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ABSTRACT 


y  We  we  interested  in  conitrnetinf  a  computer  agent  whose  behavior  will  be  intelligent  enough 
to  perform  cooperative  tasks  involving  other  agents  like  itself.  The  construction  of  such  agents  has 
been  a  major  goal  of  artificial  intelligence  research.^One  of  the  key  tasks  such  an  agent  must  perform 
is  to  form  plans  to  carry  out  its  intentions  in  a  complex  world  in  which  other  planning  agents  also 
exist.  To  construct  such  agents,  it  will  be  necessary  to  address  a  number  of  issues  that  concern 
the  interaction  of  knowledge,  actions,  and  planning.  Briefly  stated,  an  agent  at  planning  time  must 
take  into  account  what  his  future  states  of  knowledge  will  be  if  he  is  to  form  plans  that  he  can 
execute;  and  if  he  must  incorporate  the  plans  of  other  agents  into  his  own,  then  he  must  also  be  able 
to  reason  about  the  knowledge  and  plans  of  other  agents  in  an  appropriate  way.  These  ideas  have 
been  explored  by  several  researchers,  especially  McCarthy  and  Hayes  [McCarthy  and  Hayes  1960] 
and  Moore  [Moore  1980], 

Despite  the  importance  of  this  problem,  there  has  not  been  a  great  deal  of  work  in  the  area  of 

formalising  a  solution.  Formalisms  for  both  action  and  knowledge  separately  have  been  examined 

.  C  i__y 

in  some  depth,  but  there  have  been  few  attempts  at  a  synthesis.  The  exception  to  this  is  Moore ’i  1  & 

thesis  on  reasoning  about  knowledge  and  action -fMoo re  1980])for  which  a  planner  has  been  recently 
proposed  [Appelt  1980].^  Moore  shows  how  a  formalism  based  on  vssible- world  semantics  can  be 
used  to  reason  about  the  interaction  of  knowledge  and  action.  In  this  paper  we  develop  an  alternative 
formalism  for  reasoning  about  knowledge,  belief,  and  action;  we  show  how  this  formalism  can  be 
used  to  deal  with  several  well-known  problems,  and  then  describe  how  it  could  be  used  by  a  plan 
constructing  system. 
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1.  Introduction 


We  an  interested  in  constructing  a  computer  agent  whose  behavior  will  be  intelligent  enough 
to  perform  cooperative  tasks  involving  other  agents  like  itself.  The  construction  of  such  agents  has 
been  a  major  goal  of  artificial  intelligence  research.  One  of  the  key  tasks  such  an  agent  must  perform 
is  to  form  plans  to  carry  out  its  intentions  in  a  complex  world  in  which  other  planning  agents  also 
exist.  To  construct  such  agents,  it  will  be  necessary  to  address  a  number  of  issues  that  concern 
the  interaction  of  knowledge,  actions,  and  planning.  Briefly  stated,  an  agent  at  planning  time  must 
take  into  account  what  his  future  states  of  knowledge  will  be  if  he  is  to  form  plans  that  he  can 
execute;  and  if  he  must  incorporate  the  plans  of  other  agents  into  his  own,  then  he  must  also  be  able 
to  reason  about  the  knowledge  and  plans  of  other  agents  in  an  appropriate  way.  These  ideas  have 
been  explored  by  several  researchers,  especially  McCarthy  and  Hayes  [McCarthy  and  Hayes  1989] 
and  Moore  [Moore  1980]. 

Despite  the  importance  of  this  problem,  there  has  not  been  a  great  deal  of  work  in  the  area  of 
formalising  a  solution.  Formalisms  for  both  action  and  knowledge  separately  have  been  examined 
in  some  depth,  but  there  have  been  few  attempts  at  a  synthesis.  The  exception  to  this  is  Moore’s 
thesis  on  reasoning  about  knowledge  and  action  [Moore  1980],  for  which  a  planner  has  been  recently 
proposed  (Appelt  1980],  Moore  shows  how  a  formalism  based  on  possible- world  semantics  can  be 
used  to  reason  about  the  interaction  of  knowledge  and  action.  In  this  paper  we  develop  an  alternative 
formalism  for  reasoning  about  knowledge,  belief,  and  action;  we  show  how  this  formalism  can  be 
used  to  deal  with  several  well-known  problems,  and  then  describe  how  it  could  be  used  by  a  plan 
constructing  system. 


We  seek  a  formalisation  of  knowing  and  acting  such  that  a  description  of  their  interaction 
satisfies  our  intuitions.  In  the  first  section,  we  present  a  basic  formalism  for  describing  an  agent’s 
static  beliefs  about  the  world.  W«  take  a  syntactic  approach  here:  an  agent’s  beliefs  are  identified 
with  formulas  in  a  first-order  language,  called  the  object  language  (OL).  Propositional  attitudes 
such  as  knowing  and  wanting  are  modeled  u  a  relation  between  an  agent  and  a  formula  in  the  OL. 
By  introducing  a  language  (the  metalanguage,  or  ML)  whoso  prime  object  of  study  is  the  OL,  we 
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are  able  to  describe  an  agent’s  beliefs  as  a  set  of  formulas  in  the  OL,  and  express  partial  knowledge 
of  that  theory.  An  agent’s  reasoning  process  can  be  modeled  as  an  inference  procedure  in  the  OL: 
from  a  base  set  of  facts  and  rales  about  the  world,  he  derives  a  full  set  of  beliefs,  called  his  theory 
of  the  world. 

The  syntactic  approach  to  representing  propositional  attitudes  is  well-known  in  the  philosophy 
literature,  and  in  the  artificial  intelligence  field  McCarthy  [McCarthy  1979]  has  developed  a  closely 
related  approach.  The  formalism  developed  here  differs  mainly  in  that  it  explicitly  identifies  proposi¬ 
tional  attitudes  as  relations  on  sentences  in  an  object  language,  and  uses  provability  in  the  OL  as 
the  model  of  an  agent’s  reasoning  process.  We  are  able  to  present  quite  complex  deductions  involv¬ 
ing  the  beliefs  of  agents  (see  the  Wise  Man  Puss/e  in  Appendix  A,  for  example)  by  exploiting  the 
technique  of  semantic  attachment  to  model  directly  an  agent’s  reasoning  process.  We  are  indebted 
to  Weyhrauch  [Weyhraueh  1980]  for  an  introduction  to  this  technique,  and  for  the  general  idea  of 
using  ML/OL  structures  to  represent  agents. 

Finally,  our  work  differs  from  McCarthy’s  in  its  careful  axiomatisation  of  the  relation  between 
ML  and  OL,  and  incorporates  solutions  to  several  technical  problems,  including  reasoning  about 
belief- nesting  (beliefs  about  beliefs;  Creary  [Creary  1980]  has  also  described  a  solution),  and  a  cleaner 
approach  to  representing  quantified  OL  expressions  in  the  ML.  (This  latter  subject  is  not  directly 
relevant  to  this  paper,  and  will  be  reported  in  [Konolige  1981].) 

An  alternative  to  the  syntactic  approach  to  representing  propositional  attitudes  is  the  posriMe- 
mo rid  approach,  so-called  because  it  utilises  Kripke-type  possible- world  semantics  for  a  modal  logic  of 
knowledge  and  belief.  Moore  [Moore  1980]  has  shown  how  to  reason  efficiently  about  propositional 
attitudes  by  using  a  first-order  axiomatisation  of  the  possible-world  semantics  tor  a  modal  logic. 
Our  objections  to  the  possible- world  approach  are  twofold:  first,  the  possible-world  semantics  for 
representing  propositional  attitudes  is  complex  and  at  times  unintuitive;  to  deduce  facts  about  an 
agent’s  knowledge,  one  must  talk  about  the  possible-worlds  that  are  compatible  with  what  the  agent 
knows.  Ultimately,  we  suspect  that  the  syntactic  approach  will  prove  to  be  a  simpler  system  in 
which  to  perform  automatic  deduction,  but  further  research  in  both  areas  is  needed  to  decide  this 
issue.  A  second  objection  is  that  it  seems  to  bo  difficult  to  modify  possible-world  semantics  for  the 
modal  logic  to  model  adequately  inference  processes  other  than  logical  deduction.  The  possible- world 
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approach  uses  the  modal  axiom  that  every  agent  knowa  the  consequences  of  his  knowledge,  and  this  it 
obviously  not  true,  if  only  became  real  agents  hare  resource  limitations  on  their  reasoning  processes. 
The  syntactic  approach  does  not  suffer  from  this  criticism,  became  it  is  possible  to  describe  explicitly 
in  the  ML  the  inference  procedure  an  agent  might  me. 

The  second  part  of  this  paper  integrates  the  syntactic  approach  to  representing  knowledge  and 
belief  with  a  situation  calculus  [McCarthy  and  Hayes  1969]  description  of  actions.  We  concentrate 
on  many  of  the  interactions  between  knowledge  and  action  presented  in  Moore's  thesis  [Moore  1980]. 
Simply  stated,  Moore’s  account  is  that  an  agent's  beliefs  in  any  situation  arise  from  at  least  three 
sources:  direct  observation  of  the  world,  persistence  of  beliefs  about  previous  situations,  and  beliefs 
about  what  events  led  to  the  current  situation.  By  formalising  this  assumption,  he  shows  how 
to  mods!  in  an  intuitively  plausible  way  the  knowledge  an  agent  needs  to  perform  actions,  and 
the  knowledge  that  he  gains  in  performing  them.  Although  we  subscribe  to  his  notions  on  how 
knowledge  and  action  should  interact,  for  the  reasons  stated  above  we  feel  that  the  possible-world 
approach  Moore  uses  to  formalise  these  ideas,  while  elegant,  may  not  have  the  same  intuitive  appeal 
as  the  syntactic  approach. 

The  main  contribution  of  this  paper  is  to  show  that  the  syntactic  approach,  when  integrated 
with  a  situational  calculus  description  of  actions,  can  adequately  formalise  Moore’s  criteria  for  the 
interaction  of  knowledge  and  belief.  An  important  benchmark  is  to  formalise  the  idea  of  a  test:  an 
agent  can  perform  an  action  and  observe  the  result  to  figure  out  the  state  of  some  unobservable 
property  of  the  world.  We  conclude  the  second  section  with  just  such  an  example. 

In  the  final  section  we  comider  the  application  of  these  results  to  a  planning  system,  in  particular 
one  that  would  require  an  agent  to  take  account  of  other  agents’  plans  in  forming  his  own.  We 
come  to  the  conclusion  that  such  a  planning  system  may  not  be  significantly  different  from  current 
situation  calculus  planners  in  its  method  of  search,  but  does  require  considerably  more  sophistication 
in  the  deductions  it  performs  at  each  node  in  that  March. 


3 


1.  Agents'  Beliefs  and  First-Order  Theories 


In  this  section  we  ley  the  basic  groundwork  for  our  syntactic  approach  to  representing  and 
reasoning  about  agents’  beliefs.  We  will  model  an  agent’s  beliefs  about  the  world  as  a  sat  of 
statements  (or  theory)  in  some  first-order  language  with  equality.  This  is  not  to  say  that  an  agent 
actually  represents  the  world  as  a  set  of  first-order  statements;  we  are  not  concerned  here  with  the 
details  of  the  internal  representation  of  a  computer  or  human  agent  with  respect  to  its  environment. 
All  we  seek  is  a  way  of  modelling  the  beliefs  of  an  agent  in  a  manner  that  will  make  reasonable 
predictions  about  the  agent’s  behavior,  and  still  be  formally  tractable.  To  this  end  we  assume  that 
we  can  represent  an  agent’s  beliefs  about  the  world  as  a  set  of  statements  in  a  first-order  language, 
and  model  the  derivation  of  new  beliefs  by  an  agent  as  an  inference  process  on  those  statements. 

Consider  an  example  from  the  blocks- world  domain;  let  Ao  be  the  name  of  an  agent.  Ao  will 
have  some  set  of  beliefs  about  the  state  of  the  blocks-world.  We  represent  Ao’s  beliefs  as  a  list  of 
well-formed  formulas  (wffs)  in  a  first-order  language  with  equality.  We  call  this  list  of  wffk  Ao’s 
theory  of  the  world.  For  example,  suppose  Ao  believes  that  block  B  is  on  block  C,  and  that  he  is 
holding  block  D.  Then  we  would  have: 


Ao’s  Theory  of  the  Blocks- World 

ON[B,C) 

HOLDING^, D) 


where  ON  and  HOLDING  have  the  appropriate  interpretations. 

Besides  specific  facts  about  the  state  of  the  world,  Ao  also  has  some  general  rules  about  the  way 
the  world  is  put  together.  For  instance,  Ao  may  know  the  rule  that  if  any  block  x  is  on  any  block  y, 
then  the  top  of  y  is  not  clear.  Using  this  rule  together  with  specific  beliefs  about  the  world,  he  may 
be  able  to  deduce  that  C  is  not  clear.  This  can  be  modeled  as  a  process  of  extending  Ao’s  initial  set 
of  beliefs  about  the  world  to  include  the  deduced  information: 


4 


Ao’i  Facts  and  Rules  about  the  World  =*  .do’s  Theory  of  the  World 


ON(B,C) 

HOLDING^, D ) 

V*p  ON(x,  y)D~CLEAR(y) 


ON(B,C) 

HOLDING[Ao,D) 

Vzy  ON(z,  y)  D  ~CLEAR(y) 
~CLEAR{C) 


Thu*  an  agent’s  theory  of  the  world  will  be  the  closure  of  a  set  of  facts  and  rules  about  the  world, 
under  some  suitably  defined  inference  procedure.  We  will  call  the  set  of  basic  facts  and  rules  from 
which  all  other  beliefs  are  derived  the  base  set  of  the  theory.  Note  that  the  inference  procedure  that 
derives  the  consequences  of  the  base  set  need  not  be  logical  deduction;  it  is  readily  demonstrated  that 
people  do  not  know  all  the  consequences  of  their  beliefs,  that  they  derive  contradictory  consequences, 
etc.  We  recognise  that  the  problem  of  deriving  the  consequences  of  beliefs  for  more  realistic  inference 
procedures  is  a  thorny  and  unsolved  one,  and  do  not  intend  to  pursue  it  here.  For  the  purposes  of 
this  paper  we  have  chosen  logical  deduction  at  the  inferential  procedure:  an  agent  will  be  able  to 
deduce  the  logical  consequences  of  Us  beliefs. 

3.1  Metalanguage  and  Object  Language 

If  we  were  always  to  have  complete  knowledge  of  an  agent’s  beliefs,  then  it  would  be  possible  to 
use  a  simple  list  of  facts  and  rules  to  represent  the  base  set  of  those  beliefs.  However,  it  is  often  the 
case  that  our  knowledge  is  incomplete;  we  may  know  that  an  agent  either  believes  fact  P  or  fact  Q, 
but  wo  don’t  know  wUch.  Such  a  description  of  an  agent's  beliefs  cannot  be  modeled  by  a  list  of 
facts.  So  the  modelling  process  must  be  extended  to  a  description  of  an  agent’s  beliefs.  Since  beliefs 
are  wft  in  a  first-order  language,  a  metalanguage  can  be  used  to  describe  a  collection  of  such  wlb 
[Kleene  1967].  The  basic  idea  is  to  have  terns  in  the  metalanguage  to  denote  syntactic  expressions 
in  the  first-order  language  used  to  encode  an  agent’s  beliefs.  The  latter  first-order  language  is  called 
the  object  /anguage,  or  OL,  since  it  is  the  object  of  study  of  the  metalanguage  (ML).  Predicates  in 
the  metalanguage  are  used  to  state  that  an  expression  of  the  object  language  is  in  an  agent’s  theory 
of  the  world.  The  full  expressive  power  of  the  metalanguage  is  available  for  describing  a  given  theory 
of  the  object  language. 
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It  is  natural  to  choose  a  firit-ordar  language  for  the  metalanguage,  since  wo  will  be  Interested, 
in  proof  procedures  in  the  ML  as  well  as  the  OL.  Let  ML  be  a  sorted,  first-order  language  with 
variables  restricted  to  range  over  particular  sorts.  The  domain  of  discourse  of  the  ML  will  be  both 
the  syntactic  expressions  of  the  ML,  as  well  as  the  domain  of  discourse  of  the  OL.  Thus  the  ML  will 
be  able  to  state  relationships  that  hold  between  OL  expressions  and  the  actual  state  of  the  world. 

A  baric  division  of  sorts  of  the  ML  is  between  terms  that  denote  individuals  in  the  world, 
and  terms  that  denote  expressions  in  the  OL.  Among  the  former  will  be  terms  that  denote  agents 
(Ao,  At, . . .)  and  agents’  theories  of  the  world;  these  will  be  called  7/  terms.  We  will  use  the  function 
th  one  argument,  an  agent,  to  denote  that  agent's  theory  of  the  world. 

The  other  major  sort  of  terms  will  denote  formulas  of  the  OL;  these  will  be  referred  to  as  7/* 
terms.  Restricting  our  attention  for  the  moment  to  sentential  formulas  of  OL,  there  will  be  terms 
in  ML  that  denote  propositional  letters  in  OL,  and  constructors  in  ML  for  putting  together  more 
complicated  formulas  from  these  letters.  For  example,  P  in  ML  denotes  the  propositional  letter  F  of 
the  OL,1  and  the  ML  term  and(P,  Q')  denotes  the  sentence  FAQ  of  the  OL.  These  ML  constructors 
form  an  abstract  syntax  [McCarthy  1962]  for  OL  expressions. 

Writing  names  of  formulas  using  and,  or,  not,  and  imp  as  constructors  is  somewhat  cumbersome. 
For  the  most  part  we  will  use  a  syntactic  abbreviation,  enclosing  an  OL  formula  in  sense  quotes,3  to 
indicate  that  the  standard  ML  term  for  that  formula  is  intended.  For  example,  we  will  write: 

'FAQ'  for  and(P,Q>) 

'FD(QVR)’  for  imp{P,or{Q',P)) 

and  so  on 

The  rule  for  translating  sense-quote  abbreviations  into  T?  terms  of  the  ML  is  to  replace  each 

predicate  symbol  F  of  the  sense-quote  expression  by  the  ML  term  symbol  P,  and  each  boolean 

connective  by  the  corresponding  ML  boolean  constructor.  As  more  sorts  are  introduced  into  the  ML 

we  will  extend  the  sense-quote  convention  in  various  ways. 

Finally,  wo  introduce  the  ML  predicates  TRUE,  FACT,  and  PR,  each  of  which  has  an  OL 

formula  as  one  of  its  arguments.  TRUE(f),  where  /  is  an  OL  formula,  means  that  /  is  actually 

‘The  fnml  coaeeattoa  win  be  to  ise  primed  terms  la  IdL  to  denote  the  corresponding  aaprimed  formalae  ia  OL. 

’Thsp  are  called  ftnse-qxotee  to  tadkate  that  tko  sense  of  the  expression  Is  wanted,  rather  than  Its  trath-valae.  Ia 
(Kaplan  19T1|  them  are  called  Dge  qiotes. 
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true  in  the  world  under  consideration.  It  it  often  the  cue  that  we  will  want  to  describe  a  certain 
condition  actually  holding  in  the  world,  independent  of  whether  tome  agent  believes  it  or  not;  for 
instance,  this  is  critical  to  our  reasoning  about  events  in  the  next  section,  where  events  are  defined 
as  transformations  from  one  state  of  the  world  to  another. 

We  intend  TRUE  to  have  the  normal  Tarskian  definition  of  truth,  so  that  the  truth-recursion 
axioms  are  valid.  Let  the  variables  /  and  g  range  over  OL  expressions.  Then  we  can  write  the 
metalanguage  axioms  for  truth-recursion  in  the  object  language  as  follows: 


V/  ~TRUE(f)  =  TRUE(not{f)) 
V/g  TRUE(f)VTRUE{g)  =  TRUE{or(f,g)) 
Vfg  TRUE{f)ATRUE(g)  =  TRUE{and{J,g)) 
Vfg  TRU E{J)  DTRU E(g)  =  TRUE(imp(f,g)) 


(Tfl) 


FACT{t,  f),  where  t  is  an  OL  theory,  means  that  /  is  one  of  the  base  set  formulas  of  the  theory 
(and  from  which  the  rest  of  the  theory  will  be  derived  by  deduction).  Using  FACT,  agent  .do’s 
previously  exhibited  beliefs  about  the  world  could  be  described  by  the  following  ML  predicatea: 

F  ACT{th(Ao),  'ON{B,  C)’) 

FACT(th(Ao),  ‘HOLDING^,  D)’) 

FACT(th{Ac),  ‘Vip  ON(x,  y)D~CLEAR(g)’) 

The  last  FACT  predicate  describes  a  rule  that  agent  Aq  believes. 

One  special  type  of  FACT  that  we  will  make  frequent  use  of  is  a  formula  known  to  all  agents. 
We  define  the  predicate  CFACT  on  OL  expressions  to  mean  that  a  true  expression  is  a  FACT  tor 
all  agents,  that  is,  a  Common  FACT: 


V/  CFACT[f)  D  Vo  FACT{th(a),  f)ATRUE(f)  (CF1) 

CF  doesn’t  completely  axiomatise  what  we  intend  a  common  fact  to  be,  however,  since  it  doesn’t 
say  that  every  agent  knows  that  every  agent  knows  that  every  agent  knows  /,  etc*  But  a  fuller 
characterisation  of  CFACT  must  wait  until  the  technical  machinery  tor  describing  belief-nesting  is 
developed  in  a  later  subsection. 

*Ceaaoa  fade  in  asset  to  be  tbs  same  as  tbs  'uj  fool*  ooacspt  of  [McCarthy  ltrTl]. 
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PR{t,  f)  means  that  /  is  provable  in  the  theory  t.  At  discussed  previously,  we  will  assume  that 
PR  gives  the  closure  of  sentences  in  OL  that  can  be  generated  by  logical  deduction  from  an  original 
set  of  FACTi.  A  simple  axiomatisation  of  PR  can  be  given  for  Hilbert-style  (assumption-free) 
proofs.  There  is  only  one  rule  of  inference,  Modus  Ponear. 

Vt/#  imp(f,  g))APR(t,  f)  D  PR(t,  g)  (MP) 

that  is,  from  PDQ  and  P  in  the  OL,  infer  Q.  Since  every  FACT  is  an  initial  theorem  of  the  theory, 
we  assert  that  each  of  these  is  provable: 

Vtf  FACT(t,  /)  D  PR(t,  /).  (FP) 

And  in  each  theory  the  logical  axioms  of  a  Hilbert  system  need  to  be  asserted;  we  assume  a  sufficient 
set  for  the  sentential  ease. 

MP  and  the  Hilbert  axioms  will  be  used  in  ML  proofs  of  the  provability  of  OL  statements; 
these  axioms  simulate  a  Hilbert-type  proof  system  for  an  OL  theory.  This  simulation  is  necessary 
because  in  general  there  will  be  an  incomplete  ML  description  of  the  OL  theory,  rather  than  a 
simple  list  of  FACT*  for  that  theory.  In  those  special  cases  when  a  list  of  FACTi  is  available,  it  is 
possible  to  run  the  proof  procedure  on  the  OL  theory  directly.  That  is,  since  the  intended  meaning 
of  the  PR  predicate  is  provability  in  the  OL  theory,  we  can  check  whether  the  PR  predicate  holds 
in  the  ML  by  running  a  theorem- provar  in  the  OL.  It  alto  isn’t  necessary  to  use  a  Hilbert  system, 
and  we  will  feel  free  to  exploit  any  system  of  natural  deduction  that  is  sound.  The  technique  of 
using  a  computable  model  of  the  intended  interpretation  of  a  predicate  to  determine  the  truth  of 
formulas  involving  that  predicate  is  called  semantic  attachment  [Weyhraueh  1980],  and  it  will  be 
used  extensively  to  simplify  proofs  in  later  sections. 

The  provability  predicate  PR  does  not  have  the  same  characteristics  as  TRUE,  and  this  is 
important  in  representing  beliefs.  For  example,  the  fact  that  P  is  not  provable  doesn't  imply  that 
~P  is  provable.  If  we  identify  provability  with  belief,  ~*PR(th(M),  ’P*)  asserts  that  P  is  not  one  of 
A«'s  beliefs  about  the  world,  but  this  does  not  Imply  PR{th{Ao),'~P’),  i.e.,  that  Ao  believes  ~P. 
Also,  it  is  possible  to  express  that  either  Ao  believes  that  C  is  clear,  or  ha  believes  that  C  it  not 


clear: 


PR(th(Ac),  'CLEAR(Cy)VPR{th(Ao),  ‘~CLEAR{C)y, 

this  uji  something  quite  different  from  PR(th(Ao),,CLEAR(C)V~CLEAR{C)’)-l  the  latter  is  a 
tautology  that  every  agent  believes,  while  the  former  says  something  a  lot  stronger  about  Ac’ s  beliefs 
about  the  world. 

Paralleling  the  truth  recursion  axioms  TR,  we  can  state  rules  for  the  provability  of  compound 
OL  expressions  in  terms  of  their  immediate  subexpressions.  Because  of  the  nature  of  provability, 
the  axioms  for  negation,  disjunction,  and  implication,  unlike  their  truth-theoretic  counterparts,  are 
not  equivalences. 


Vt/~PR(t,/)CPf?(t,n«rt(/)) 
Vt/y  [P«(t,/)vP«(t,y)l  D  PR(t,or{f,g)) 
Vtfg  [PR(t,  f)APR(t,  g)]  =  PR(t,  a nd{f,  g)) 
Vtfg  [ PR(t,f)DPR(t,g )]  C  PR{t,imp{f,g)) 


(PR) 


These  are  all  deducible  from  the  logical  axioms  in  the  Hilbert  proof  system;  for  instance,  the  last 
assertion  is  just  a  restatement  of  Modus  Poaeas. 

Another  interesting  connection  between  the  PR  and  TRUE  predicates  can  be  drawn  by  looking 
at  models  of  the  OL.  Suppose  we  have  used  FACT  and  PR  to  describe  an  agent’s  theory  T  of  the 
world.  There  will  be  some  set  of  models  that  satisfy  T,  i.e.,  for  which  all  of  7”s  theorems  hold. 
The  actual  world  will  be  one  of  these  models  just  in  case  all  7”s  theorems  hold  for  the  world.  This 
condition  is  statable  in  the  ML  as: 


V/  PR{T,f)DTRUE{f) 

In  general  this  assertion  will  not  be  valid,  that  is,  an  agent’s  beliefs  need  not  correspond  to  the  actual 
world.  By  introducing  the  predicate  TRUE  in  the  ML,  we  are  able  to  state  the  correspondence 
between  a  given  theory  of  the  world  and  the  actual  state  of  affairs  in  the  world. 

Li  Knowledge  aad  Belief 

The  PR  and  TRUE  predicates  can  be  used  to  state  our  fundamental  definitions  of  knowing 
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and  believing  for  an  agent.  BEL(a,f)  meant  that  agent  a  beHet'es  /;  KNOW[a,f)  mean*  that 
agent  a  know*  /.  Then  we  have  the  definition!: 


Vo/  BEL(a,  f)  =  PR(th(a),  f) 

Va/  KNOW{a,f)  =  BEL(a,  f)/\TRUE{f) 


(Bl) 


That  ii,  we  identify  belief  with  provability  in  an  OL  theory,  and  knowledge  at  a  belief  that  actually 
holdi  in  the  woild.  In  model-theoretic  tenni,  a  lentence  it  known  to  an  agent  if  the  nntence  holdi 
in  all  of  hit  models,  and  the  actual  world  it  a  model  for  that  sentence.  The  definition  of  a  common 
fact  in  CF 1  means  that  all  common  facts  are  known  to  all  agents. 

We  already  know  that  the  inference  process  used  in  deriving  new  beliefs  from  old  ones  is  only 
approximated  as  logical  consequence,  yet  we  should  still  expect  this  approximation  to  correctly 
model  some  of  the  characteristics  we  attribute  to  belief.  For  instance,  if  a  rational  agent  believes  that 
PDQ,  and  he  doesn’t  believe  Q,  then  it  should  be  the  case  that  he  doesn’t  believe  P.  Translating 
to  the  above  notation  yields  the  sentence: 


BEL(Ao, ‘PDQ')A ~BEL(Ac, ‘Q’)  D  ~BEL{Ao,‘P’) 

To  illustrate  the  use  of  axioms  for  belief  and  provability  given  so  far,  we  exhibit  a  natural  deduction 
proof  of  this  sentence  in  ML: 


1. 

BEL{Ao,‘PDQ’) 

given 

2. 

PRm*>).'PDQ’) 

1,51 

3. 

•^BEUAo/Q') 

given 

4. 

-PR(tA(Ao),‘Q’) 

3, SI 

5. 

PR(th(Ao),  ‘P’)  D  PR[th(Ao),  ‘Q’) 

2, PR 

8. 

~PJ?(tMAo),,P’) 

4,5  contrapositive 

T. 

~BEL(Aq,  ’P’) 

6, SI 

This  particular  proof  in  the  ML  cannot  be  done  by  semantic  attachment  to  the  OL,  because  it 
involves  reasoning  about  what  isn’t  provable  in  the  OL  theory. 

At  this  point  we  have  presented  the  basic  ideas  and  definitions  for  a  syntactic  approach  to 
representing  and  reasoning  about  agents’  beliefs.  The  rest  of  this  section  is  devoted  to  exploring 
various  technical  issues  that  vise  when  extending  the  previous  analysis  to  talking  about  individuals. 
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U  bdhiduli 


By  restricting  ourselves  to  the  cue  of  sentential  formulas  in  OL,  w*  hare  been  able  to  present 
the  basic  concepts  for  representing  the  beliefs  of  an  agent  more  simply.  Additional  complications 
arise  when  dealing  with  terms  in  the  OL  that  denote  individuals  rather  than  truth-ralues.  But  a 
ML  encoding  of  these  terms  is  necessary  in  order  to  express  such  concepts  as  agent  Ao  knows  who 
B  is. 

To  talk  about  the  individuals  that  the  OL  refers  to,  we  introduce  an  additional  sort  into  the 
ML,  whose  denotation  will  be  the  function  terms  of  the  OL.  This  sort  will  be  called  Tr,  and  consists 
of  the  following  members: 

(1)  variables  a,p,...; 

(2)  {/*(ti, . . .,  t»)>,  where  t,  €  Tr, 

(TT) 

(3)  *>(t),  where  t  €  Tr  [the  "standard  name*  functionj; 

(4)  nothing  else. 

The  ML  variables  a,  ft,  . . .,  range  over  OL  function  terms.  For  example,  we  can  state  that  Ao 
believes  a  particular  block  is  on  C  by  asserting  the  ML  expression: 

3a  BEHAcOrfia,?)). 

In  this  expression  there  are  two  ML  terms  in  7Y,  namely,  a  and  C1.  C'  is  a  O-ary  function  (or 
constant)  in  Tr  that  denotes  the  constant  term  C  in  OL.4  ON'  is  a  type  of  ML  term  that  hasn’t 
been  used  explicitly  before;  it  is  a  member  of  Tr  because  it  names  an  OL  formula.  It  takas  two 
arguments,  each  of  which  is  a  ML  term  denoting  an  OL  term,  and  constructs  an  OL  formula  that  is 
the  OL  predicate  ON  of  these  arguments.  So  the  ML  term  ON1  [a,  C*)  denotes  the  OL  expression 
ON(A,  C),  where  A  is  the  OL  term  denoted  by  a. 

It  is  now  possible  to  give  a  full  definition  of  Tr  terms: 

(1)  variables  .; 

(2)  {/"(ti... •,*»)}.  where  6  7V  [boolean  constructors,  e.g.,  and]; 

[TF) 

(3)  . . .,  tn))i  where  6  Tr  [predicate  constructors,  e.g.,  ON *J; 

(4)  nothing  else. 

*We  cxternd  the  prime  eeaveattoa  to  caver  ML  terms  la  IV  as  well  as  7>;  that  to,  I*  la  ML  deaotes  the  aaprtoed 
term  t  la  OL. 


mwWfry ft/'  *■*■■* •■* 


11 


and  T[  term*: 


(1)  variables  *,y,...; 

(2)  {/"(tii  •  •  -i  t»)},  when  t,  g  Tt  [indiridnal  constants  and  functions]; 

( TI) 

(3)  A(t),  whan  t  6  Tt  [the  denotation  function]; 

(4)  nothing  else. 

We  will  also  find  it  convenient  to  extend  the  notion  of  sense-quote  abbreviations  to  handle  ML 
terms  involving  Tt  variables.  The  previous  rules  are  expanded  in  the  following  way:  ail  function 
symbols  in  the  sense-quote  expression  are  replaced  by  their  primed  forms,  while  any  symbols  used 
as  variables  in  the  surrounding  ML  expression  remain  unchanged.  For  example,  the  sense-quote 
expression  in  3aKNOW[Aa,  ‘ON [a,  6(C))')  is  to  be  understood  as  a  syntactic  abbreviation  for 
the  ML  term  ON'(a,  V[C')).  We  have  not  yet  said  what  happens  to  T/  variables  in  sense-quote 
expressions;  this  must  wait  until  standard  names  are  explained  in  the  next  subsection. 

The  introduction  of  Tt  terms  into  the  ML  completes  the  descriptive  power  of  ML  for  OL 
expressions.  It  also  lets  us  handle  some  of  the  weil-known  denotational  pussies  in  the  philosophy 
literature.  One  of  the  simplest  of  these  is  the  Morningstar-Eveningstar  description  problem.  Both 
Morning  star  and  Eveningstar  are  actually  the  planet  Venus  seen  at  different  times  of  the  day.  An 
agent  Ao  believes  that  they  are  not  the  same;  further,  he  doesn’t  have  any  knowledge  about  either 
being  the  planet  Venus.  Let  MS,  ES,  and  VENUS  be  OL  terms  that  denote  the  Morningstar,  the 
Eveningstar,  and  Venus,  respectively.  The  following  set  of  ML  formulas  describes  this  situation: 

TRUSSES  =  VENUS’) 

TRUE(’MS  =  VENUS’ ) 

BEHAo/MS  ft  ES1) 

~BEHAq,‘ES  =  VENUS’) 

~BEHAo,'MS  =  VENUS’) 

It  is  perhaps  easiest  to  explain  this  set  of  sentences  in  model-theoretic  terms.  The  intended  inter¬ 
pretation  of  the  the  OL  terms  ES,  MS,  and  VENUS  it  the  tame  object,  namely  the  planet  Venus. 
The  two  TRUE  predicates  establish  this,  since  they  assert  that  these  three  terms  denote  the  same 
individual  in  the  world.  On  the  other  hand,  the  first  BEL  predicate  asserts  that  in  the  models  of 
A«’s  theory  of  the  world,  MS  and  ES  denote  different  individuals.  This  means  that  the  actual 


world  cannot  b«  among  the  models  of  this  theory.  Farther,  the  last  two  BEL  predicates  assert  that 
ES  and  MS  are  not  provably  equal  to  VENUS  in  this  theory;  hence  there  will  he  some  models 
of  the  theory  for  which  ES  =  VENUS  holds,  some  for  which  MS  =  VENUS  holds,  and  some 
for  which  neither  holds.  From  this  we  conclude  that  not  only  is  Ao  mistaken  as  to  the  equality  of 
ES  and  MS,  he  also  is  unsure  about  whether  either  is  the  same  at  VENUS.  [McCarthy  1979}  lists 
tome  other  philosophical  pussies  that  can  be  handled  in  a  syntactic  formulation. 

2.4  Knowing  Who  Someone  b 

One  of  the  problems  that  any  formal  treatment  of  belief  must  confront  is  that  of  describing 
when  an  agent  knows  who  or  what  something  is.  For  example,  the  following  two  English  sentences 
say  something  very  different  about  the  state  of  Ac's  knowledge:* 

(1)  "An  knows  who  murdered  John.” 

(2)  *Ao  knows  that  someone  murdered  John.” 

The  police  would  certainly  be  interested  in  talking  to  Ao  if  the  first  statement  were  true,  while  the 
second  statement  just  means  that  Ao  read  the  local  tabloid.  We  might  paraphrase  the  first  statement 
by  saying  that  there  is  some  individual  who  murdered  John,  and  Ao  knows  who  that  individual  is. 
The  second  statement  can  be  true  without  Ao  having  any  knowledge  about  the  particular  individual 
involved  in  the  murder. 

How  is  the  distinction  between  the  two  sentences  above  to  be  realised  in  this  formalismf  The 
second  sentence  is  easy  to  represent: 

BEL(Ao,  ‘3*  MURDERED[x,  JOHN)’)  (Wl) 

This  simply  sty*  that  Ao  believes  in  the  existence  of  an  individual  who  murdered  John.  It  might  be 
supposed  that  the  first  sentence  could  be  represented  in  the  following  way: 

3a  BEL{Ac,‘MURDERED(a,JOHN)')  (W2) 

Wl  says  that  there  is  a  MU RDERED  predicate  in  Ao ’s  theory  of  the  world  relating  some  individual 
(a’s  denotation)  and  John.  Unfortunately,  this  isn’t  quite  strong  enough;  if  the  denotation  of  a  is 

*A  (hollar  problem  appeal*  la  (Qsia*  1ST1] 
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the  OL  term  murdtrtr(JOHN),  then  W2  it  virtually  a  tautology,  and  doesn’t  say  that  Ac  know* 
who  murdered  John.  Indeed,  if  the  OL  expression  in  Wl  it  ikolemised,  it  become*  obvious  that  Wl 
and  W2  are  equivalent. 

What  *eem*  to  be  going  on  here  is  that  different  name*  have  a  different  itatu*  at  far  a*  identifying 
individual*  is  concerned.  'Bill*  is  a  sufficient  description  for  identifying  John's  murderer,  whereas 
■John’s  murderer*  is  not.  The  question  of  what  constitutes  a  sufficient  description  is  still  being 
debated  in  the  philosophical  literature.  But  for  the  purposes  of  this  paper,  it  will  suffice  if  we  have 
a  name  that  it  guaranteed  to  denote  the  tame  individual  in  every  model  of  the  OL.  By  asserting  a 
predicate  involving  this  name  in  do’s  theory  of  the  world,  it  will  be  possible  to  encode  the  fact  that 
Ac  believes  that  predicate  for  the  given  individual.  Names  that  always  denote  the  tame  individual 
are  called  standard  names. 

The  formal  method  of  establishing  standard  names  is  straightforward.  Consider  the  set  of 
all  individuals  involved  in  the  situation  we  wish  to  consider.'  Include  in  the  OL  a  set  of  constant 
symbols,  the  standard  name  ijmbolt,  to  be  put  in  one-one  correspondence  with  these  individuals. 
The  language  OL  will  be  partially  interpreted  by  specifying  this  correspondence  as  part  of  any  model 
of  the  language;  this  means  that  the  only  models  of  OL  we  will  consider  are  those  that  are  faithful 
to  the  standard  name  mapping. 

In  the  metalanguage,  we  introduce  the  standard  name  function  ij  of  one  argument  (tee  the 
definition  of  Tj  terms  above).  This  function  returns  the  standard  name  of  its  argument.  Generally 
we  will  use  lower  case  Greek  letters  from  the  later  part  of  the  alphabet  as  ML  variables  for  OL 
standard  names  (/i,  v,  . . .].  The  metalanguage  statement  of  "Aq  knows  who  the  murderer  of  John 
it*  then  becomes: 


3*ji  (iK*)  =  H)  A  KNOW[Ac,  ' MURDERED(fi ,  JOHN)')  (W3) 

Because  it  denotes  a  standard  name,  the  only  models  of  this  statement  are  those  in  which  the  same 

individual  z  murdered  John.  This  it  in  contrast  to  Wl  and  W2  above,  which  allow  models  in  which 

any  individual  murdered  John.  An  immediate  consequence  it  that  Wl  and  W2  are  derivable  from 

W3,  but  not  the  other  way  around. 

•We  nitric*  eemhres  to  esemUMs  sets  here. 
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So  in  order  to  assert  that  Ac  know*  who  or  what  some  individual  B  it,  we  write  in  the  ML:T 
3*/i  (ij(x)  =  n)  A  K  NOW{Ao,  ‘B  =  n‘) 

By  modifying  the  leme-qoote  tranilation  rnlei  slightly,  it  ii  pouibie  to  write  OL  expressions 
involving  standard  names  much  more  compactly.  The  modification  is  to  assume  that  any  ML 
variable  of  type  7/  occurring  within  a  sense-quote  gets  translated  to  the  standard  name  of  that 
variable.  With  this  rule,  for  example,  the  above  assertion  comes  out  as  3x  KNOW(Ac,‘B  =  x’). 

We  will  use  the  predicate  KNOWIS[a,p)  to  mean  that  the  agent  a  knows  who  or  what  the 
OL  term  denoted  by  P  refers  to.  The  definition  of  KNOWIS  is: 

Va0  KNOWIS(a,  p)  =  3*  KNOW[a,  ‘p  =  z’)  (KW) 

Note  that  the  property  of  being  a  standard  name  is  a  relation  between  a  term  of  the  OL  and  models 
of  this  language,  smd  hence  cannot  be  stated  in  the  OL.  The  use  of  a  metalanguage  allows  us  to  talk 
about  thx  relation  between  the  OL  and  its  models. 

One  of  the  proof-theoretic  consequences  of  using  standard  names  is  that  every  theory  can  be 
augmented  with  inequalities  stating  the  uniqueness  of  individuals  named  by  standard  names.  In  the 
metalanguage,  we  write: 


(SN) 


Formally,  the  definition  of  a  standard  name  can  be  axiomatised  in  the  ML  by  introducing 
the  denotation  function  A.1  A(a),  where  a  denotes  an  OL  term,  is  the  denotation  of  a  in  the 
actual  world;  it  is  the  inverse  of  the  standard  name  function,  since  it  maps  an  OL  term  into  its 
denotation.  There  is  an  intimate  relation  between  the  denotation  function  and  equality  statements 
in  OL  formulas  describing  the  world: 

TThl*  aaaly*!*  eweetlally  follow*  that  of  [Ksplaa  Tlj,  with  tho  extension  of  rtaidard  nomoo  to  all  ladividaal*  la  tho 
domain,  rath**  tkaa  ja*t  anmbon  aad  a  few  othor  abstract  object*.  Thor*  an  problems  la  wise  rtaadard  aam**  for 
complex  Individual*,  however;  no  [Kaplan  Tl|. 

•This  I*  Church’*  doaotatloa  pndlcato  la  faactioa  form  [Church  1951);  line*  a  term  caa  have  only  oaa  denotation,  it 
la  aimpler  to  u*o  a  faactioa. 
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'iafi  TRUE('a  -  0')  =  A(a)  =  A(/J) 


(Z?l) 


that  is,  two  OL  terms  are  equal  in  the  actual  world  just  in  case  they  denote  the  same  individual;  D 1 
can  be  viewed  as  a  definition  of  the  intended  interpretation  of  equality.  The  prime  purpose  of  the 
denotation  function  is  to  tie  together  the  denotation  of  terms  in  the  OL  and  the  ML.  For  standard 
names,  it  can  be  used  to  state  that  the  denotation  of  a  standard  name  is  the  same  individual  in  all 
situations,  something  that  cannot  be  done  with  equality  predicates  in  the  OL: 

V*  A(i*(*))  =  *  (D2) 

For  example,  by  asserting  tj(VENUS )  =  VENUS '  in  ML,  we  fix  the  denotation  of  the  OL  term 
VENUS'  to  be  the  individual  denoted  by  the  ML  term  VENUS  in  all  models  of  the  OL. 

The  introduction  of  standard  names  with  fixed  denotations  across  all  models  makes  the  task  of 
relating  the  OL  to  the  ML  easier.  By  introducing  this  “common  coin*  for  naming  individuals,  we 
are  able  to  write  expressions  of  the  OL  that  represent  beliefs  without  constantly  worrying  about  the 
subtle  consequences  of  the  denotational  variance  of  terms  in  those  expressions.  Standard  names  will 
play  an  important  role  in  describing  belief-nesting  (beliefs  about  beliefs),  in  describing  executable 
actions,  and  in  simplifying  the  deduction  process. 

3.5  The  Object  Language  as  Metalanguage 

In  this  subsection  we  extend  the  OL  to  include  a  description  of  another  object  language  OL'. 
Thus  extended,  the  OL  can  be  viewed  as  a  metalanguage  for  OL'.  The  reason  we  want  to  do  this 
is  that  it  will  be  necessary  for  representing  an  agent’s  view  of  a  world  that  it  changing  under  the 
influence  of  events.  In  the  next  section  we  will  show  how  an  agent  can  model  the  way  in  which  the 
world  changes  by  describing  what  is  true  about  different  states  of  the  world  connected  by  events. 
But  to  describe  these  states  of  the  world,  or  situations,  the  agent’s  theory  must  talk  about  sentences 
of  another  language  holding  in  a  given  situation. 

Before  trying  to  extend  the  formal  apparatus  of  the  OL  to  describe  another  OL,  it  is  helpful  to 
examine  more  closely  the  relation  between  the  ML  as  a  means  of  studying  the  OL  and  as  a  means 
of  describing  the  actual  world.  This  is  because  the  structure  of  a  ML/OL  pair  will  be  very  similar 
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no  matter  what  the  depth  of  embedding;  and  the  limpleat  inch  itructnre  to  study  ii  obviously  the 
topmost  one.  Although  we  initially  characterised  the  ML’s  domain  of  discourse  as  including  that 
of  the  OL,  it  appears  that  we  have  not  made  much  use  of  this  characterisation.  In  describing  the 
models  of  OL,  however,  it  was  necessary  to  pick  out  the  model  that  was  the  actual  world;  this  was 
done  with  the  predicate  TRUE.  And  it  was  impossible  to  state  the  definition  of  a  standard  name 
without  appealing  to  terms  in  the  ML  that  referred  to  individuals  in  the  actual  world.  So,  in  fact, 
we  have  already  used  the  ML  to  characterise  the  actual  state  of  the  world  and  the  individuals  that 
populate  it. 

We  have  stated  that  agents’  beliefs  are  represented  as  first-order  theories  of  the  world.  The 
ML  is,  by  the  above  argument,  just  such  a  theory;  but  whose  theory  of  the  world  is  itT  One  useful 
interpretation  is  to  taka  what  we  will  call  the  egocentric  view;  a  theory  in  the  ML  is  identified  as 
the  theory  of  a  particular  agent.  That  is,  suppose  we  were  to  build  a  computer  agent  and  invest 
him  with  a  ML/OL  structure  as  a  way  of  representing  other  agents’  beliefs.  Then  the  nonlogical 
axioms  of  the  ML  would  constitute  the  computer  agent’s  theory  of  the  world.  The  interpretation 
of  the  ML  predicate  TRUE  would  be  “what  the  computer  agent  believes  about  the  world,'*  and 
of  the  predicate  KNOW,  “what  another  agent  believes  that  agrees  with  what  the  computer  agent 
believes.”  In  this  interpretation,  there  is  no  sense  of  absolute  truth  or  knowledge;  the  beliefs  of  one 
agent  are  always  judged  relative  to  those  of  another. 

Suppose  we  identify  the  agent  Ao  with  the  ML;  what  interpretation  does  the  OL  theory  th(Ao) 
have?  Interestingly  enough,  it  is  Ao’s  introspective  description  of  his  own  beliefs.  Unlike  other 
agents'  theories  of  the  world,  th(Ao)  shares  an  intimate  connection  with  formulas  that  hold  in  the 
ML.  For  a  rational  agent,  it  should  be  the  case  that  if  ho  believes  P,  then  ha  believes  that  he  believes 
P.  We  can  state  this  connection  by  the  following  rule  of  inference; 

Belief  attachment:  If  the  agent  a  is  identified  with  the  ML,  then  from 
TRUE(f)  infer  BEL(th(a),  f). 

Introspection  will  bo  useful  when  we  consider  planning,  because  a  planning  agent  must  be  able  to 
reflect  on  the  future  state  of  his  beliefs  when  carrying  out  some  plan. 

If  the  metalanguage  is  intended  to  describe  the  actual  world,  then  it  is  reasonable  to  ask  what 
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the  relation  ii  between  models  of  the  ML  and  models  of  its  OL,  and  whether  this  connection  can 
be  formalised  in  the  ML.  We  start  by  adding  predicate  symbols  to  the  ML  whose  intended  meaning 
is  a  property  of  the  actual  world,  rather  than  of  the  OL  and  its  models.  Consider  such  a  predicate 
P  of  no  arguments,  and  let  its  intended  meaning  be  "222  Baker  Street  Apt  13  is  unoccupied:*  that 
is,  the  actual  world  satisfies  P  just  in  case  this  apartment  is  indeed  unoccupied.  In  the  OL  there 
is  also  a  predicate  symbol  P  of  no  arguments  whose  meaning  we  wish  to  coincide  with  that  of  the 
ML  predicate  P.  The  fact  that  these  symbols  are  the  same  is  an  orthographic  accident;  they  come 
from  different  languages  and  there  is  thus  no  inherent  connection  W  them.  However,  because 
the  ML  can  describe  the  syntax  and  semantics  of  the  OL,  it  i»  fR.*s»te  a  axiomatise  the  desired 
connection.  Let  P*  be  the  ML  term  (in  7»  denoting  the  OL  P.  Then  P  in  the  ML  and 

OL  have  the  same  meaning  if 

P=TRUE(P)  (ill) 

is  asserted  in  the  ML.  For  suppose  the  actual  world  satisfies  P  in  the  ML;  then  TRUEiP1)  must 
also  hold,  and  hence  by  the  meaning  of  TRUE,  the  actual  world  is  also  a  model  for  P  in  the  OL. 
Similarly,  if  the  actual  world  falsifies  P  in  the  ML,  TRUElno^P1))  must  hold,  and  the  actual  world 
falsifies  P  in  the  OL  also.  So  the  proposition  named  by  P*  holds  just  in  case  Apt.  13  at  222  Baker 
Street  is  unoccupied,  and  thus  the  meanings  of  P  in  the  ML  and  P  in  the  OL  coincide. 

For  predicates  that  have  arguments,  the  connection  is  complicated  by  the  need  to  make  sure 
that  the  terms  used  in  the  ML  and  OL  actually  refer  to  the  same  individuals.  So,  for  example,  if 
P  is  a  ML  predicate  of  two  arguments  that  we  wish  to  mean  the  same  as  the  OL  predicate  P,  we 
would  write: 

Va/J  TRUE('P(a,fly)=P(A(a),  A(fl);  (J?2) 

that  is,  since  the  denotation  function  A  gives  the  individuals  denoted  by  the  OL  terms  a  and  ft, 
P  in  the  ML  agrees  with  P  in  the  OL  on  these  individuals.  Using  standard  names,  R2  could  be 
rewritten  as 

V*yP(f,p)srKUff(,P(z,p)’)  (P3) 

since,  by  D2,  A(iK*))  =  *,  A(iKv))  *  f  Note  that  the  standard  name  convention  for  sense-quotes 


ii  in  force  for  A3. 


Using  TRUE  end  equivalence,  axioms  like  A3  cause  predicate  symbols  to  hare  a  "standard 
meaning*  acroti  the  ML  and  OL,  in  much  the  tame  way  that  D2  formalises  itandard  name*  using 
the  denotation  function  and  equality.  But  while  nonstandard  names  are  a  useful  device  for  encoding 
an  agent's  beliefs  about  individuals  that  the  agent  may  have  misidentifled  (recall  the  Morningstar- 
Eveningstar  example),  nonstandard  predicates  don't  seem  to  serve  any  useful  purpose.  So  we  will 
assume  that  for  every  predicate  symbol  P  in  ML,  there  is  a  function  symbol  of  the  form  P1  whose 
denotation  is  the  OL  predicate  P,  and  there  is  an  axiom  of  the  form  R2  equating  the  meaning  of 
these  predicates. 

To  make  the  OL  into  a  metalanguage  for  OL1,  we  simply  introduce  sorts  that  denote  OL' 
expressions  into  the  OL,  in  exactly  the  same  way  that  it  was  done  for  the  ML.  In  addition,  the 
various  axioms  that  tie  the  ML  and  OL  together  (MP,  Dl,  etc.)  must  also  be  asserted  in  the  OL. 
Unfortunately,  this  also  means  that  the  ML  itself  must  have  a  new  set  of  terms  denoting  terms  in 
the  new  OL  sorts;  the  machinery  for  describing  embedded  ML/OL  chains  rapidly  becomes  confusing 
as  the  depth  of  the  embedding  grows.  So  in  this  paper  we  will  supply  just  enough  of  the  logical 
machinery  to  work  through  the  examples  by  introducing  two  conventions;  readers  who  want  more 
detail  are  referred  to  [Konolige  1981]. 

The  first  convention  is  an  extension  of  the  sense-quote  abbreviation  to  include  ML  variables 
of  the  sort  7>  (denoting  formulas  of  the  OL).  When  these  occur  in  sense-quotes,  they  are  to  be 
translated  as  the  standard  name  of  the  variable;  hence  they  denote  the  name  of  an  expression.  To 
taka  an  example,  we  will  complete  the  axiomatisation  of  CFACT: 

V/  CFACT{  f)  D  Va  KNOW(a,  ,CFACT{f)’)  (CF2) 

CF 2  asserts  that  if  /  is  a  common  fact,  then  every  agent  knows  it  is  a  common  fact.  The  sense-quote 
term  'CFACT(f)'  denotes  the  OL  expression  CFACT(ft),  where  f  is  the  standard  name  of  the 
OL'  expression  corresponding  to  /. 

The  second  convention  is  to  allow  embedded  sense-quotes  to  form  the  itandard  name  of  an 
expression,  as  in 
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V/  CFACT(‘CFACT(f)D  CFACT{,CFACT{f)>)').  (CF3) 

Hon  the  embedded  mum  quotas  translate  to  the  standard  nama  for  tha  OL'  exprassion  CFACT(P). 
CF3  says  that  common  facts  will  ba  inherited  down  to  the  next  level  of  embedding  in  tha  ML/OL 
chain. 

In  practice,  we  hope  that  the  depth  of  embedding  needed  to  solve  a  given  problem  will  be  small, 
since  tha  complexity  needed  for  even  the  three-level  structure  of  ML,  OL,  and  OL*  is  substantial. 
Also,  the  technique  of  Mmantic  attachment  can  be  used  to  reduce  the  complexity  of  reasoning  about 
embedded  structures  by  attaching  to  a  particular  level  of  an  embedded  structure  and  reasoning  in 
that  language.  In  Appendix  A  we  use  embedded  ML/OL  structures  to  solve  the  wise  man  pusxle, 
which  involves  reasoning  to  a  depth  of  embedding  of  three  (ML,  OL,  and  OL');  we  exploit  semantic 
attachment  to  simplify  the  reasoning  involved. 
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3.  The  Interaction  o t  Actio ni  and  Beliefs 


The  previous  (action  laid  the  groundwork  for  a  syntactic  treatment  of  knowledge  and  belief  in 
a  static  world.  Thii  most  be  integrated  with  a  formal  treatment  of  actions  in  order  to  acenmpiieli 
onr  original  talk  of  form  alii  ing  the  interaction  of  knowledge  and  action.  We  the  following 

two  questions: 

•  What  knowledge  ii  required  by  an  agent  to  successfully  perform  an  action? 

•  What  knowledge  does  an  agent  gain  in  performing  an  action? 

The  methodology  we  will  sue  is  to  apply  the  situation  calculus  [McCarthy  and  Hayes,  1969]  approach 
to  first  formally  describe  the  way  in  which  the  world  changes  as  events  occur.  It  will  then  be  mnm*H 
that  this  formal  system  is  a  reasonable  approximation  to  the  way  an  agent  reasons  about  changes 
in  the  world:  this  means  that  it  becomes  part  of  an  agent’s  rules  about  the  world.  By  simply 
attributing  a  facility  for  reasoning  about  events  to  agents,  it  turns  out  that  we  are  able  to  answer 
both  these  questions  formally,  and  that  this  formalisation  corresponds  well  with  our  intuitions  about 
real  agents.  This  is  essentially  the  same  method  that  was  used  by  Moore  in  [Moore  1980];  here,  we 
show  that  it  can  be  successfully  carried  out  for  a  syntactic  formalisation  of  knowledge  and  belief. 

Once  the  formal  requirements  for  reasoning  about  events  have  been  specified,  we  consider  how 
an  agent  might  plan  to  achieve  a  goal  using  his  knowledge  of  actions.  We  conclude  that  planning  is 
inherently  a  process  of  self-reflection:  that  is,  in  order  to  construct  a  plan,  an  agent  must  reflect  on 
what  the  state  of  his  beliefs  will  be  as  the  plan  is  undergoing  execution.  Such  a  self-reflection  process 
is  represented  naturally  by  a  ML/OL  structure  in  which  the  planning  agent  is  identified  with  the 
ML,  and  his  future  states  are  theories  of  the  OL.  We  will  show  how  it  is  possible  to  construct  plans 
within  this  representation,  and  extend  it  to  include  plans  that  involve  other  cooperative  agents. 

3.1  Situations 

In  the  situation  calculus  approach,  events  are  taken  to  be  relations  on  situations,  where  situa¬ 
tions  are  snapshots  of  the  world  at  a  particular  moment  in  time.  It  is  natural  to  identify  situations 
with  models  of  a  language  used  to  describe  the  world;  in  this  case,  we  will  use  the  language  OL  of 
the  previous  section,  because  the  ML  for  describing  models  of  the  OL  is  already  laid  out.  In  the 
ML,  situations  will  be  named  by  terms,  generally  the  constants  {So,  5t, . . .}.  A  formula  /  of  the 
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OL  holds  in  &  situation  s  when  the  situation  satisfies  /;  the  ML  predicate  H{»,f)  will  be  used  to 
indicate  this  condition.  If  the  situation  S0  is  singled  out  as  being  the  actual  world  (and  the  initial 
world  for  planning  problems),  then  TRUE  can  be  defined  in  terms  of  H: 

V/  TRUE(/)=H(S0,  /)•  (ffl) 

Since  H  describes  satisfiability  in  a  model,  the  truth-recursion  axioms  TR  are  valid  for  H  as  well  as 
TRUE. 

If  we  consider  agents  to  be  part  of  the  domain  of  discourse,  then  their  beliefs  can  change  from 
one  situation  to  the  next,  just  as  any  other  inessential  property  of  an  agent  might.  But  if  an  agent’s 
beliefs  change  from  situation  to  situation,  then  the  theory  that  is  used  to  model  these  beliefs  must 
also  change.  One  way  to  represent  an  agent’s  changing  beliefs  is  to  ascribe  a  different  theory  to  an 
agent  in  each  situation  to  model  his  beliefs  in  that  situation.  In  the  ML,  we  will  write  tht(a,  #)  to 
denote  agent  a’s  beliefs  in  situation  s;  if  50  is  taken  to  be  the  actual  world,  then  it  is  obvious  that 

Va  thi{a,  So)  =  th(a). 

But  we  might  now  ask  what  situation  the  expressions  in  each  of  these  theories  are  about. 
Suppose  that  the  OL  sentence  P  is  a  member  of  th*[Ao,  Si),  and  thus  one  of  *>’s  beliefs  in  situation 
Si.  We  would  naturally  want  P  to  be  a  property  that  A*  belieres  to  hold  of  situation  (and  not  S0 
or  some  other  situation).  That  is,  tht(a,  t)  represents  agent  a’s  beliefs  in  situation  »,  about  situation 
t.  In  informal  usage  we  will  call  the  situation  we  are  focusing  on  the  current  situation,  and  say  “the 
agent  a  in  situation  s’  when  we  we  referring  to  the  agent’s  beliefs  in  that  situation.  Later  we  will 
show  how  to  represent  an  agent’s  beliefs  about  situations  other  than  the  one  he  is  currently  in. 

For  each  situation,  an  agent's  beliefs  in  that  situation  are  specified  by  a  theory.  Given  this 
arrangement*  we  define  the  new  predicates  B  and  K  as  similar  to  BEL  and  KNOW,  but  with  a 

situation  argument: 

Vos/  B(o, »,  /)  a  PR(ths(a,  s),  /)  ^ 

Vos/  K(  a,  t,  f)  =  B{a,  i,  f)SE{$,  f) 

B(a,  s,  /)  means  that  in  situation  s  agent  a  believes  that  /  holds  in  s;  K  is  similar,  with  the  condition 
that  /  actually  holds  in  s.  Note  that  the  underlying  predicates  FACT  and  PR  do  not  have  to  be 
rince  they  are  defined  on  theories  of  OL  rather  than  models.  Thus  the  properties  of  BEL 
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and  KNOW  described  in  the  previous  section  also  bold  for  B  and  K  in  a ny  particular  situation. 
BEL  and  KNOW  can  be  defined  as  B  and  K  in  the  situation  So. 

Several  extensions  to  the  formalism  presented  in  the  first  section  must  be  made  to  deal  with 
situations.  A  new  denotation  function  8  takes  a  situation  argument  as  well  as  an  OL  term:  6(t,  a)  is 
the  denotation  of  a  in  situation  i.  A(a)  gives  the  denotation  of  a  a  in  situation  So,  and  is  definable 
as  f(So,  a).  The  appropriate  forms  of  D\  and  D2  are: 
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Vtap  H(‘a  =  fi’)  =  [«(*,£»)  =  8{t,fi)\ 

Vst  s(i,  t »(*))  =  * 

This  last  says  that  standard  names  always  have  the  same  interpretation  in  every  situation.  Non¬ 
standard  names  can  change  their  denotation  in  different  situations,  e.g.,  the  block  denoted  by  "the 
block  Ao  is  holding”  may  be  changed  by  Ao‘s  actions. 

Finally,  we  require  the  appropriate  versions  of  Rl-RS,  where  these  axioms  are  appropriately 
generalised  to  refer  to  all  situations. 


3.3  Observables 

Following  Moore  [Moore  1980],  we  recognise  three  ways  that  an  agent  can  acquire  beliefs  in  a 
situation: 

•  He  can  observe  the  world  around  him. 

•  His  beliefs  about  past  situations  persist  in  the  current  situation. 

•  He  can  reason  about  the  way  in  which  the  current  situation  arose  from  events  that  occurred 
in  previous  situations. 

In  the  next  few  subsections  we  describe  how  an  agent’s  beliefs  persist  and  how  he  reasons  about 
events;  here  we  formalise  what  it  meant  for  a  property  of  the  world  to  be  observable. 

It  is  certainly  true  that  there  are  many  properties  of  the  world  we  live  in  that  are  not  directly 
observable;  for  example,  consider  a  gas  oven  whose  pilot  light  is  completely  encased  and  hence 
not  visible.  Whether  this  pilot  light  is  on  or  off  isn’t  an  observable  property,  but  there  are  other 
observations  that  could  be  made  to  test  what  the  state  of  the  pilot  light  is,  e.g.,  by  turning  on  the 
oven  and  observing  whether  it  lights.  What  we  actually  consider  to  be  observable  depends  on  how 
we  formalise  a  given  problem  domain;  but  it  it  important  for  a  planning  agent  to  be  able  to  maka 
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the  distinction  between  properties  of  the  world  he  can  observe  directly,  and  those  he  most  infer. 


One  of  the  reasons  that  it  is  handy  to  have  a  separate  theory  representing  the  beliefs  of  an 
agent  in  each  situation  is  that  we  then  have  a  way  of  describing  the  effect  of  observable  properties 
on  an  agent’s  beliefs.  Formally,  we  can  state  that  a  property  is  observable  by  asserting  that  in 
every  situation,  subject  to  certain  preconditions  that  are  required  for  the  felicitous  observation  of 
the  property,  an  agent  knows  whether  that  property  holds  or  not.  For  example,  in  the  OL  let  o  be 
an  oven,  and  let  LIT{o)  mean  that  o  is  lit.  Then  LIT[o)  it  asserted  to  be  observable  by: 


Vaos  H(s,‘AT(a,  o)’)  D  [  K(a,  s,  ‘LIT{o)')\lK{<x,  t,  ‘~LIT(oY)  ]  (Ol) 

that  is,  if  the  agent  is  actually  at  the  oven,  he  knows  either  that  it  is  lit,  or  that  it  is  not  lit.  Recall 
from  the  previous  section  on  knowledge  and  belief  that  this  says  something  very  strong  about  the 
state  of  a’s  knowledge,  and  is  not  derivable  from  the  tautology  K{a,  t,  ‘LIT(o)V~LIT(o)’). 

U  Events  Types 

Event  types  are  relations  on  situations;  a  given  event  type  describes  the  possible  states  of  the 
world  that  could  result  from  an  event  occurring  in  any  initial  state.  We  will  use  the  three-place 
predicate  EV  in  the  metalanguage  to  describe  event  types:  EV(e,  »,-,*/),  where  <  is  an  event  type 
and  »i  and  sy  are  situations,  means  that  if  results  from  an  event  of  type  t  occurring  in  s,-.  An  event 
is  an  instance  of  an  event  type,'  but  generally  we  will  not  have  to  distinguish  them  for  the  purposes 
of  this  paper,  and  we  will  use  “event”  for  “event  type*  freely. 

Generally  the  events  of  interest  will  be  agents'  actions,  and  these  will  be  constructed  in  the 
ML  using  terms  representing  actions,  agents,  and  the  objects  involved  in  the  action  (the  parameters 
of  the  action).  If  act  is  an  action,  then  do{a,  act)  is  the  event  of  agent  a  performing  this  action. 
Consider  the  situation  calculus  axiomatisation  of  a  simple  blocks-worid  action,  puton{x,  y),  where 
the  parameters  of  the  action  are  blocks: 

■Tw  tump  It,  *B«tg*t  wtaaiag  of  Wimbltdoa  jtrttrday  vu  fortaltow*  Is  a  statsmeat  aboat  a  siagii  want,  bet  *Beig 
wtsalag  Wimbltdoa  bat  happsaad  Iva  Mints'  diatribes  as  mat  typo  that  bad  Bvs  particular  Instances. 
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Vatftiif  EV(do(a,puton(s,  p)),  a,  #,}  s  ff(s{,  ‘CLEAJUpflA 

mt{,'HOLDING[a,zY)  A 
H{»f,'ON{z,py)  a  1  ; 

ff(s,,  ‘~HOLDING{a,  x)*)A 

Vazptit/  EV(do(aj>uton{z,  p)),  s<,  »f)  D 

[  V/  SAF(f)Af  *  ‘CLEARLY A f  5*  ‘ HOLDING (a,  *)'  D  (P02) 

«(•<,/)=%,/)] 

The  form  of  POl  ii  an  equivalence,  10  the  right- hand  tide  describes  the  conditions  under  which 
situations  s,  and  s/  are  related  by  the  event  of  a  putting  z  on  p.  The  first  two  conjuncts  on  the  right- 
hand  side  are  essentially  preconditions  for  the  erent  to  occur,  since  they  state  conditions  on  the  initial 
situation  s<  that  must  be  satisfied  for  EV  to  hold.  The  preconditions  are  that  CLEAR[ri{p))  and 
HOLDING(ti(a),  tl(z))  must  hold  in  situation  s,-;  note  that  the  standard  names  for  the  parameters 
are  indicated  by  the  sense-quote  convention.  If  the  preconditions  are  not  met,  then  there  it  no 
situation  s/  that  is  the  successor  to  under  the  erent  e.  The  rest  of  the  conjuncts  describe  which 
formulas  of  the  OL  are  to  hold  in  the  new  situation  •{. 

P02  specifies  that  all  formulas  of  a  certain  type  that  hold  in  s<  are  also  to  hold  in  */.  It  is  thus 
a  frame  axiom  for  the  erent  e,  describing  which  aspects  of  the  situation  n  remain  unchanged  after 
the  erent  occurs.  The  predicate  SAF  stands  for  Simple  Atomic  Formula;  it  picks  out  those  formulas 
of  the  OL  that  are  composed  of  atomic  predicates  orer  standard  names.  Although  SAF  applies  only 
to  nonnegated  atomic  formulas,  the  frame  axiom  carries  orer  negated  atomic  formulas  as  well,  since 
H(i,not(f))  is  equivalent  to  ~H(s,/).10  Among  the  nicer  features  of  this  axiomatisation  is  that 
events  whose  outcomes  are  conditional  on  the  initial  state  can  be  easily  described.  For  instance, 
consider  the  erent  of  an  agent  turning  on  a  gas  oven  that  has  a  pilot  light.  If  the  pilot  light  is  on,  the 
oven  will  be  lit;  if  the  pilot  light  is  off,  the  oven  will  hare  whatever  status,  lit  or  unlit,  it  had  before 
the  erent  occurred  (the  oven  may  already  have  been  on).  Let  PL(o)  be  an  OL  predicate  meaning 
“the  pilot  light  of  oven  0  it  on*;  and  let  LIT[o)  mean  ‘oven  0  is  lit*  Then  the  erent  of  an  agent 
turning  on  0  can  be  described  as: 


laThe  ixiomattsattoi  of  swats  given  ken  Is  s  etaadard  oao  la  tke  AI  H  two  tan  oa  formal  plaaalag,  sad  Ikon  an 
weil-kaowa  problems  lawIrlBC  tko  an  of  frame  axioms  Uko  tko  oao  skew.  Wo  an  aot  attempting  to  odd  tap  sow 
is  rick t  to  tkls  partiealar  aspect  of  p  lean  tic;  bat  wo  an  latenetod  la  karlac  s  formal  dascriptioa  of  swats  to  latsgrate 
wttk  ear  theory  of  belief,  and  this  seems  to  bo  the  boot  formats  ttoa  camatty  available. 
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Vas<s fO  EV {doia.ligh^o)),  s<,0/)  = 
ff(*<,‘AT(«,o)’)  A 
H{ti,'PL{oy)DH(if,'LIT{oy )  A 
H(s,,*~PI(o)’)  D  |H(./,'L/T(o)')=H(i<,'L/r(»n| 


£V(do(o,Jt#M(o)),  $i,  if)  D 

IV/  SAF(f)Af  7*  ‘  Wr(»)'  3  H(«i,  /)=»(*/,  /)  ]  1  J 

The  second  conjunct  of  LT1  gives  the  result  of  the  event  in  cue  the  pilot  light  is  on:  the  oven  will 
be  lit.  The  third  conjunct  ujt  that  if  the  pilot  light  is  off,  the  oven  will  be  lit  in  </  just  in  case  it 
was  lit  in  it,  i.e.,  its  status  doesn’t  change.  LT2  is  the  frame  axiom. 


3.4  Reasoning  about  Situations  and  Events 

The  axiomatiiation  of  events  as  relations  on  situations  enables  ns  to  talk  about  what  is  true  in 
the  world  after  some  events  have  occurred  starting  from  an  initial  situation  (which  we  will  generally 
take  to  be  So).  What  it  doesn’t  tell  us  is  how  an  agent’s  beliefs  about  the  world  will  change;  nothing 
in  the  PO  or  LT  axioms  gives  any  insight  into  this.  It  might  be  suspected  that,  as  events  are 
described  by  axioms  as  changing  the  actual  state  of  the  world,  this  description  might  be  extended  to 
cover  agents’  theories  u  well,  e.g.,  changing  4o's  theory  in  situation  5b  (ths(4>.  So))  into  his  theory 
in  situation  Si  (thi(Ao,  Si)).11  But  there  is  no  obvious  or  well-motivated  way  to  main  modifications 
to  axioms  like  PO  and  LT  so  that  they  take  into  account  agents’  beliefs  about  a  situation  rather 
than  what  actually  holds  in  the  situation.13  •  •'hat  is  needed  here  is  a  principled  way  of  deriving 
the  changes  to  an  agent’s  beliefs  that  result  from  an  event,  given  a  description  of  the  event  as  a 
relation  on  situations.  Credit  for  the  recognition  of  this  problem  belongs  to  Robert  Moore,  and  we 
will  formalise  the  solution  he  presented  in  hit  thesis,  the  main  points  of  which  follow. 

The  solution  to  this  difficulty  lies  in  making  the  observation  that  agents  are  reasoning  entities. 
Consider  how  agent  Ao  might  reason  about  tome  event  5;  let  us  suppose  the  event  is  that  agent  Ao 

tuned  on  the  oven  in  situation  5o,  and  that  the  result  was  that  the  oven  was  not  lit  in  situation  Si. 

nIadesd,  it  might  be  tboeght  that  the  meet  widely  kaowa  AI  plsaalag  system,  STRIPS,  has  Jast  sack  a  mechaaJsm 
la  the  Its  add/delete  Hat  approach  to  describing  rrsats.  However,  doeer  examination  reveals  that  beeaase  STRIPS 
makes  the  airsmptios  that  it  has  a  partial  model  la  the  sease  of  [Weyhraach  leeoj,  It  h  actually  sOghtty  leer  descriptive 
thaa  the  situational  approach  described  above  [Nilsson  1900|. 

l*Thsso  Is  oae  proposal  that  b  reggested  by  the  ear  iso  of  B  to  refer  to  the  actual  attention  aad  PR  to  statements 
that  aa  agent  believes  about  a  situation,  lamely,  to  replace  all  predicates  tavslvimg  H  with  the  corresponding  oaee 
laveMag  PR.  However,  it  eaa  be  shewn  that  the  suhetitulioa  of  PR(tKt(A*,  s), . . .)  for  U (s, . . .)  yields  couaterintalttve 
results  for  Ao’i  beliefs. 
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What  should  Ao’s  belief*  be  in  situation  Si?  Pint,  by  observation,  he  knows  that  the  oven  isn't  lit. 
He  also  believes  (in  Si)  that  the  current  situation  resulted  from  the  event  E  occurring  in  situation 
So.  So  Aa  reasons  as  follows:  if,  in  situation  So,  the  pilot  light  of  the  oven  had  been  on,  then  in  Si 
the  oven  would  be  lit,  since  he  turned  it  on.  But  the  oven  isn’t  lit;  hence  the  pilot  light  couldn’t 
have  been  on  in  So,  and  remains  not  on  in  Si. 

There  are  several  important  things  to  note  about  this  analysis.  The  first  is  that,  u  suggested 
previously,  Ao’s  beliefs  in  situation  Si  comes  from  only  three  sources:  observations  (“the  oven  is  not 
lit*),  persistence  of  beliefs  about  previous  situations  (“if  in  So  the  pilot  light  had  been  on. . .”),  and 
beliefs  about  the  way  events  change  the  world.  This  latter  is  equivalent  to  having  some  form  of  PO 1 
as  part  of  Ao’t  beliefs  in  situation  Si.  From  these  three  sources  Ao  is  able  to  generate  a  new  set  of 
beliefs  for  Si- 

The  second  thing  to  note  is  that  none  of  Ao’t  reasoning  in  Si  could  have  taken  place  unless  he 
believed  that  Si  resulted  from  So  via  the  event  E.  Beliefs  about  what  sequence  of  events  led  to  the 
current  situation  play  a  very  important  role  in  reasoning  about  that  situation,  and,  like  other  beliefs, 
they  can  be  mistaken  or  inferred  from  other  evidence.  Suppose,  for  example,  that  Ao  suddenly  sees 
the  oven  become  lit.  He  might  infer  that  the  only  way  that  could  happen  when  it  wasn’t  previously 
lit  would  be  for  an  agent  to  turn  it  on;  this  is  inferring  that  the  situation  where  the  oven  is  lit  is 
connected  by  a  certain  event  with  a  previous  situation  where  the  oven  wasn’t  lit.  We  will  not  be 
concerned  with  this  kind  of  inference  here,  although  we  note  the  possibility  of  doing  event  recognition 
in  this  framework.  The  events  we  are  interested  in  are  actions,  and  the  assumption  we  will  make  for 
the  remainder  of  this  paper  is  that  an  agent  knows  what  action  it  is  that  he  performs  in  executing 
a  plan. 

A  third  aspect  of  this  reasoning  that  is  unusual  is  that  the  axiomatisation  of  events  is  being  used 
in  a  different  way  than  a  planning  program  would  normally  consider  doing.  Typically,  a  planner 
uses  an  event  description  like  POX  to  form  plans  to  light  the  oven,  and  the  side  condition  that  the 
pilot  light  be  on  is  one  of  the  things  that  can  go  wrong  with  the  plan,  and  so  must  be  taken  into 
account  as  a  subgoal.  However,  in  the  above  example  Ao  has  used  POX  to  reason  about  a  property 
of  the  world  that  is  not  available  to  his  direct  observation,  that  is,  as  a  test.  This  is  an  important 
characteristic  for  any  formalism  that  combines  a  description  of  agents’  beliefs  with  a  description  of 
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events;  a  single  description  of  an  event  should  suffice  for  an  agent  to  reason  about  it  either  as  a 
means  of  effecting  a  change  in  the  world,  or  as  a  test  that  adds  to  his  beliefs  about  the  world. 

Finally,  the  precondition  that  Ao  be  at  the  oven  to  turn  it  on  translates  naturally  in  this  analysis 
into  a  precondition  on  Ao’s  beliefs  in  situation  So.  If  Ao  is  to  reason  that  situation  Si  is  the  successor 
to  So  under  the  event  E,  he  must  believe  that  he  was  actually  at  the  oven  in  situation  So-  For  if  he 
doesn’t  believe  this,  then  he  cannot  use  POX  to  infer  anything  about  the  results  of  his  action. 

We  might  summarise  the  analysis  of  this  section  in  the  following  way:  by  making  the  simple 
assumption  that  an  agent  reasons  about  the  way  in  which  situations  are  related  by  events,  we  are 
able  to  characterise  in  a  natural  way  the  belief  preconditions  required  for  executing  an  action,  and 
the  effects  of  actions  on  the  subsequent  belief  state  of  an  agent.  The  interaction  of  observation  and 
reasoning  about  situations  gives  an  agent  the  power  to  plan  actions  that  perform  tests,  as  well  as 
change  the  state  of  the  world. 

3.5  Formalising  Agents’  Reasoning  about  Events 

We  now  give  a  formalisation  that  implements  the  ideas  just  laid  out.  The  first  requirement  is 
that  we  be  able  to  describe  an  agent  a  in  situation  s  reasoning  about  other  situations,  especially 
the  one  just  preceding.  Since  the  formulas  of  tho( o,  <)  all  refer  to  properties  of  situation  s,  we  must 
enrich  the  OL  so  that  formulas  in  the  OL  can  refer  to  different  situations.  Using  the  techniques 
of  belief-nesting  of  the  previous  section,  we  add  to  the  OL  the  predicate  H  corresponding  to  the 
ML  predicate  of  the  same  name.  Then  the  OL  expression  H(Si,‘P ’)  means  that  the  OL*  formula 
P  holds  in  situation  Si,  regardless  of  what  theory  this  formula  appears  in.1*  With  the  addition  of 
the  H  predicate  to  the  OL,  the  notion  that  all  formulas  in  th»( a,  s)  refer  to  properties  of  •  can  be 
formalised  as: 


V*/  PR(th(a,  s),  f)  ~  PR{th{a,  s),  ’if (»,  /)’)  (H2) 

H2  can  be  paraphrased  by  saying  that  an  agent  believes  P  in  situation  s  just  in  case  he  believes  that 
P  holds  in  situation  s.  Given  H2,  it  is  possible  to  describe  agents’  theories  as  consisting  purely  of 

formulas  in  H;  but  the  added  level  of  embedding  puts  this  technique  at  a  disadvantage  with  respect 

1*W«  win  take  (9«,  5i, . . .}  to  be  standard  aamee  fer  stteatloai  la  an  laagaafee.  It  wfll  be  aeeaaed  that  standard 
nausea  are  always  used  te  mass*  situations. 
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to  using  other  predicates  from  OL  to  deecribe  kb  agent’s  belief*  about  the  current  situation. 

It  is  also  possible  to  formalise  the  notion  that  beliefs  about  previous  situations  persist,  or  are 
carried  over  into  succeeding  situations.  Suppose  that  in  situation  Sn  an  agent  has  a  belief  of  the 
form,  “in  a  previous  situation  S<,  P  was  true.”  Then  if  Sn+t  is  the  successor  to  Sn  under  some 
event,  this  belief  is  still  valid.  Formally,  vie  can  assert  this  with  the  ML  axiom: 

Vj<s/e  EV(t,  »i,  »f)  D  \  {H3) 

The  antecedent  of  the  implication  says  that  »,■  and  »j  must  be  connected  by  some  event  for  beliefs 
to  be  carried  over  from  s<  to  </;  this  is  necessary  because  we  don’t  want  agents  to  inherit  beliefs 
from  their  future  states.  By  phrasing  the  beliefs  in  termi  of  the  predicate  H,  HZ  carries  over  beliefs 
about  all  situations  previous  to  and  including  s,-. 

One  of  the  consequences  of  HZ  is  that  once  an  agent  forms  a  belief  about  a  situation,  he  holds 
that  belief  about  that  situation  for  all  time.  Since  beliefs  can  be  mistaken,  it  might  happen  that  an 
agent  observes  something  that  forces  him  to  revise  his  previously  held  beliefs.  In  that  case,  HZ  is 
too  strong,  and  the  resultant  theory  will  be  inconsistent.  We  recagniie  that  the  general  problem  of 
reconciling  inconsistent  beliefs  that  arise  from  different  sources  (called  belief  revision)  is  a  hard  one, 
involving  both  conceptual  and  technical  issues,  and  it  is  not  part  of  this  research  to  say  anything 
new  about  it.14  Nevertheless,  it  is  worthwhile  to  note  that  because  the  ML  has  terms  that  refer 
to  agents’  theories  in  different  situations,  it  may  be  possible  to  describe  a  belief  revision  process 
formally  in  the  ML. 

3.8  An  Example  of  a  Test 

Given  the  preceding  techniques  for  describing  what  an  agent  believes  to  hold  in  situations  other 
than  the  one  he  is  currently  in,  we  can  show  formally  that  Ap  can  use  the  LT  axioms  as  a  tost  to 
figure  out  whether  the  pilot  light  is  on  or  not.  In  the  initial  situation  So,  we  will  assume  that  Ao 
knows  ho  is  at  the  oven  O  (where  O  is  the  standard  name  for  the  oven),  and  realises  that  it  is  not 
lit: 

,4Doyle  (Doyle  ISTSj  worked  oa  tki»  problem  aider  the  nbrie  'Trots  Mala  testa  oe,*  sad  mere  rectal  work  la 
aeaaeaetoak  Tteseaiag  site  coatUen  this  problem. 
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Initial  Conditions  in  the  ML 

(1)  K(Ao,S0,,AT(Ao,O)A~LIT(O)')  given 

(2)  K(A0,Sl>‘EV(do[An,light(0)),S0,3l)t)  given 

The  style  of  proof  we  will  exhibit  will  be  natural  deduction,  with  assumption  dependencies  noted 
in  square  brackets  in  the  justification  for  a  line  of  the  proof.  Given  the  initial  conditions,  we  next 
show  that  Ac  can  observe  whether  or  not  the  oven  is  lit  in  situation  Si : 


(3)  V/  SAF(f)Aj  *  lLIT{Oy  D  H(S0,  f)=H{Suf)  2fl2±T2 

(4)  SAF{,AT(Ac,0)‘)  definition  of  SAF 

(5)  H{Su'AI{Ac,OY)  1,3,4,  B2 

(8)  K(Aq,  Si,‘LIT{0)')VK[Ac,  Su‘~UT(OY)  5,01 

Line  3  comes  from  the  frame  axiom  for  light,  and  lets  us  infer  that  Ac  is  still  at  the  oven  in  situation 
Si  (line  5).  The  observation  axiom  01  is  then  invoked  to  assert  that  Ao  will  know  what  the  state 
of  the  oven  is  in  that  situation. 

Throughout  this  proof,  we  will  be  interested  in  two  theories  of  the  OL:  ths(Ac,  So)  and  th${Ac,  Si). 
Assertions  in  the  ML  involving  Ac's  beliefs  can  be  reasoned  about  by  using  semantic  attachment  to 
the  appropriate  OL  theory.  For  example,  line  1  above  is  attached  to  the  following  statements  in 
*MAo.  So): 


Ac' s  Theory  in  Situation  S0 

(7)  AT(Ao,0)A~L/T(0)  1,B2, semantic  attachment 

(8)  msQ,,~LIT(0)‘)  1,B2,H 2, semantic  attachment 

Line  7  is  the  attachment  of  Une  1  to  Ao’*  theory  in  So.  Line  8  is  derived  from  line  1  by  the  use  of  H2; 
it  is  useful  because  it  will  persist  as  a  belief  in  the  successor  situation  Si.  Generally,  beliefs  that  an 
agent  derives  about  the  current  situation  can  be  inherited  into  succeeding  situations  by  expressing 
these  beliefs  with  the  H  predicate. 
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At  this  point  we  do  reasoning  by  cases.  First  assnzne  the  right  disjunct  of  line  6;  then  for  Ao’s 


beliefs  in  situation  Si  we  have: 


Ao's  Theory  in  Situation  Si 

(9)  ~L/T(0) 

[9]  :assumed, semantic  attachment 

(10)  ~H(Si,‘LIT{0)') 

[9]:9,tf  2semantic  attachment 

(11)  EV{do(Ao,lighHO)),So,Si) 

2, semantic  attachment 

(12)  mSo.'PHOYteHiS^LinOY) 

11,LT1 

(13)  ~H(So,‘PL(OY ) 

[9]:  10,12  contrapositive 

(14)  H[So,,~PL[0)’) 

[9):13,rR  for  H 

The  first  part  of  the  result  is  derived  by  line  14,  namely,  that  if  Ao  observes  that  the  O  is  not  lit  in 
situation  So,  then  he  knows  that  the  pilot  light  was  not  on  in  situation  So-  This  sequence  of  steps 
is  interesting  because  it  illustrates  the  intermixture  of  proof  techniques  in  the  ML  and  OL.  Lines 
9,  10,  and  11  come  from  statements  in  the  ML  about  th»(Ao,Si).  Line  10  is  derived  from  line  9  in 
the  ML  by  the  application  of  axiom  H 2.  Line  11  says  that  Ao  believes  that  5t  is  the  result  of  the 
ligkt(0)  action  occurring  in  So,  and  follows  directly  from  line  2  and  semantic  attachment.  Line  12 
follows  from  line  11  and  the  event  axiom  LT1;  it  is  assumed  that  Ao  believes  this  axiom.  Finally, 
13  and  14  follow,  given  that  the  truth-recursion  axioms  for  H  are  made  available  in  all  theories  in 
the  OL. 

The  left  disjunct  of  line  6  can  be  reasoned  about  in  the  following  way  (since  lines  11  and  12  did 
not  involve  any  assumptions,  they  can  be  used  in  this  part  of  the  proof  also): 
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j4q’i  Theory  in  Situation  Si 

(15)  LIT(0 )  [15j:assumed,  iem.  ntt. 

(16)  HiSi/LITiO)')  (15]:15,H2,Mm.  att. 

(17)  ~H(So,‘L/r(O)0  3,H3,TR  for  ,sem.  att. 

(18)  ~[if(5i,  *L/r(O)')=H(S0,  ‘1/7(0)’)]  [15):18,17 

(19)  H(Sb,*~PL(0)’)  3  H(Si,‘IJT(0)’)=/r(Sb,‘LJT(0)’)  11.LT1 

(20)  ~jy(5b,‘~ PI(O)’)  [15]:18,19  contraposition 

(21)  H(Sq,‘PL(0)’)  (15]:20,TH  for  H 


Here  again,  the  first  few  lines  (15,  16  and  17)  are  established  by  reasoning  at  the  ML  about 
tht(Ac,  Si).  Line  17  comes  from  an  instance  of  axiom  H3,  which  enables  an  agent’s  beliefs  to 
persist  through  a  sequence  of  situations.  Line  19  comes  from  .Ao’s  knowledge  of  LT 1,  and  line  20  is 
the  key  step:  it  establishes  that  under  the  assumption  of  O  being  lit  in  Si,  the  pilot  light  was  on  in 
So-  Finally,  the  frame  axiom  LT2  will  carry  the  pilot  light’s  status  in  S0  forward  into  5j: 


j4o’s  Theory  of  Situation  Sx 

(22)  V/  SAF(f)Af  *  'LIT(OY  D  H(S0,  /)stf(Si,  /)  11.LT1 

(23)  SAFTPL(O)’)  definition  of  SAF 

(24)  tf  (So,  ‘PL{Oy)==H(Si,'PL(oy)  22,23 

(25)  PL{0)  [15]:21,24,J?2 

(26)  —PL(O)  (91:14,24,772 

Line  25  is  under  the  assumption  of  the  left  disjunct  of  line  6,  and  line  26  is  under  the  right  disjunct. 
In  the  ML  we  can  derive  several  results  from  the  preceding  proof  structure: 


In  the  ML 

(27)  S(i4o,Sl,’PL(0)’)V5(^o,Si,’~PI(0)’)  8,20,21 

(28)  BMo,  Su  •UT{0)’)  DBMo.Si,  ’PL(O)’)  15,25 

(29)  B(Ao,Su'~UT{0)')  D  B(A',Su'~PL(0)’)  9,26 
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Line  27  says  that  in  51(  Ac  will  either  believe  that  the  pilot  light  ii  on,  or  he  will  believe  that  it  not 
on.  Thu,  by  performing  the  action  of  lighting  the  oven,  Ac  gain*  knowledge  about  the  state  „  a 
unobservable,  the  pilot  light.  Thia  ii  the  desired  result  of  agent  Ac  using  LT1  to  perform  a  test  of 
an  unobservable  property. 

Lines  28  and  29  give  proof-theoretic  analogues  to  the  LT  axioms,  which  described  the  event  of 
lighting  the  oven  solely  in  terms  of  the  actual  situations  before  and  after  the  event.  These  assertions 
show  how  the  beliefs  of  Ao  change  under  the  influence  of  the  event  do(a,  light{0)).  By  suitably 
generalising  the  preceding  proof,  it  can  be  shown  that  28  and  29  hold  for  ail  agents  and  initial 
situations: 


Voo«{S/  EV{do(a,  light{o)),  s<, « /)AfC(a,«<,  ‘AT{a,  o)A~L/T(o)’)  D 

B(e,  if,  • LIT{oY )  D  B(a,  ifl  'PL(o)’)  (LT3) 

B(a,  if,  ’~I/T(o)*)  D  B(e,  if,  '~PL(o)’) 

LT3  is  valid  under  the  condition  that  LT1  is  assumed  to  be  believed  by  all  agents.  LT3  is  one 
description  of  the  way  in  which  an  agent’s  beliefs  change  in  a  situation  that  results  from  an  oven¬ 
lighting  event;  it  would  be  most  useful  to  a  planner  as  a  lemma  to  be  invoked  if  the  state  of  the 
pilot  light  were  to  be  tested  as  a  step  in  a  plan.  Another  lemma  about  oven-lighting  that  would  be 
useful  to  a  planner  would  be  one  in  which  the  belief  preconditions  to  an  action  were  made  explicit; 
this  would  be  used  to  plan  actions  that  light  the  oven. 

3.7  Plans  and  Planning 

In  the  previous  subsection  we  saw  how  to  characterise  the  changes  to  an  agent’s  beliefs  produced 
by  his  observations  of  events.  In  this  subsection  we  will  consider  how  to  use  these  results  as  part  of 
the  deductions  that  an  agent  needs  to  do  to  construct  workable  plans,  i.e.,  plans  that  will  accomplish 
their  goals. 

Consider  how  an  agent  might  go  about  constructing  workable  plans.  Using  hit  description  of 
various  events  (PO,  LT,  and  others)  he  can  try  to  find  a  sequence  of  actions  that  lead  to  the  desired 
goals  being  true  in  tome  final  situation.  If  we  identify  the  planning  agent  with  the  ML,  then  a  plan 
would  be  a  sequence  of  situations  connected  by  actions  performed  by  that  agent,  such  that  the  goals 
are  true  in  the  final  situation.  This  doesn’t  seem  to  Involve  the  planning  agent  in  any  reasoning 
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about  hii  b«liefi;  all  1m  needs  to  do  is  describe  how  the  actual  world  changes  under  the  influence  of 
his  actions. 


This  isn't  the  whole  story,  though.  The  plan  that  it  derived  must  be  an  executable  plan;  that 
is,  if  the  plan  is  a  sequence  of  actions,  the  agent  must  be  able  to  execute  each  of  those  actions  at 
the  requisite  time.  For  instance,  the  action  description  light(oven(J okn))  will  not  be  executable  if 
Ao  doesn’t  know  which  oven  is  John's.  For  a  plan  to  be  executable  by  an  agent,  the  agent  must 
know  what  action  is  referred  to  by  each  of  the  do-terms  in  the  plan.  According  to  a  previous  section, 
this  means  that  the  agent  must  have  the  standard  name  for  the  action  in  his  theory.  But  what 
are  standard  names  for  actions?  Following  Moore  [Moore  1980],  we  take  the  viewpoint  that  actions 
can  be  analysed  as  a  general  procedure  applied  to  particular  arguments,  e.g.,  put  on  is  a  general 
procedure  for  putting  one  block  on  top  of  another,  and  puton(A,  B)  is  that  procedure  applied  to 
the  two  blocks  A  and  B.  If  we  assume  that  all  agents  know  what  general  procedure  each  action 
denotes,  then  the  standard  names  for  actions  are  simply  the  terms  formed  by  the  action  function 
applied  to  the  standard  names  of  its  parameters.18  The  condition  that  actions  be  executable  forces 
the  planning  agent  to  make  the  critical  distinction  between  his  beliefs  at  planning  time  and  his 
beliefs  at  execution  time.  A  planning  agent  may  not  know,  as  he  forms  his  plan,  exactly  what 
action  a  particular  do-term  in  his  plan  denotes;  but  if  he  can  show  that  at  the  time  he  is  to  execute 
that  action,  he  will  know  what  it  is,  then  the  plan  is  an  executable  one.  Plans  of  this  type  occur 
frequently  in  common-sense  reasoning;  consider  a  typical  plan  Ao  might  form  to  tell  someone  what 
time  it  is.  The  plan  has  two  steps:  first  Ao  ’will  look  at  his  watch  to  find  out  what  the  time  is,  and 
then  he  will  communicate  this  information  to  the  requester.  At  planning  time,  Ao  doesn’t  really 
knew  what  the  second  action  is,  because  he  doesn’t  know  the  time,  and  the  time  is  an  important 
parameter  of  the  communication  act.  Yet  he  can  reason  that  after  looking  at  his  watch,  he  will 
know  the  time;  and  to  the  plan  is  a  valid  one. 

By  this  argument,  an  agent  must  analyse  at  planning  time  what  the  future  states  of  his  beliefs 

will  be  as  he  executes  the  plan.  Thus  the  planning  process  intrinsically  forces  the  agent  into 

1>ActasUr,  the  eoadMoa  that  ths  ptrtotUn  be  standard  semes  Is  too  strong.  Standard  aamee  have  the  property 
that  every  agent  kaews  whether  two  Individuals  named  by  ttaadard  earns*  are  the  same  er  set  la  every  tttuatloa,  but 
this  can  dittos  Is  aet  strictly  necessary  for  as  actios  te  be  executable.  Consider  the  action  of  requesting  Information 
from  the  telephone  operator;  surely  It  le  aet  required  that  aa  agent  be  able  te  dtfbreu tints  the  operator  from  every 
other  Individual  la  his  betters.  B  he  were  to  dial  the  operates  ea  two  separate  oocaatoas,  he  weald  aet  necessarily  be 
able  to  tall  if  he  talked  te  the  tame  operator  er  aet. 
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introspection  about  his  future  beliefs.  Since  we  bare  identified  the  planning  agent  ■with  the  ML, 
it  is  natural  to  represent  his  future  beliefs  during  the  execution  of  the  plan  as  OL  theories  in  the 
situations  that  the  planning  process  gives  rise  to.  If  the  planning  agent  is  Ac,  then  these  theories  are 
tht(Ac,  So)  (the  initial  situation),  tht[Ac,  Si),  etc.,  where  each  of  the  S<  results  from  its  predecessor 
via  the  execution  of  the  next  action  in  the  plan.  Ao's  planning  process  is  basically  a  simulation  of 
the  plan’s  execution  in  which  he  reasons  about  the  changes  that  both  the  actual  world  and  his  set  of 
beliefs  will  undergo  during  the  course  of  the  plan’s  execution.  By  figuring  out  what  his  future  states 
of  belief  will  be,  he  can  decide  at  planning  time  whether  an  action  of  the  plan  will  be  executable. 

For  Ac  to  take  other  agents’  plans  into  account  in  forming  his  own,  he  must  be  able  to 
represent  their  future  states  of  belief,  in  addition  to  his  own.  But  this  doesn’t  involve  any  additional 
representational  complexity,  since  Ac  is  already  keeping  track  of  his  own  beliefs  during  the  simulated 
execution  of  the  plan.  In  [Konolige  and  Nilsson  1980]  several  an  example  of  a  multiagent  plan  is 
presented;  currently  we  are  working  on  formalising  such  plans  in  the  framework  presented  here. 

Actually,  this  planning  process  bears  a  strong  resemblance  to  typical  implementations  of  a 
situation  calculus  approach  to  planning  (Warren  1974].  In  these  systems,  events  are  axiomatised 
along  the  lines  of  PO  and  LT,  and  the  planner  searches  for  a  sequence  of  situations  that  leads  to 
the  goal  by  doing  theorem-proving  with  the  event  axioms;  the  search  space  is  essentially  the  same  in 
either  approach.  The  main  difference  is  in  the  relative  complexity  of  reasoning  that  the  two  planning 
systems  must  be  able  to  handle.  In  the  approach  described  here,  the  effect  of  actions  on  the  agent’s 
beliefs  in  each  situation  greatly  increases  the  deductive  complexity  of  the  planner  and  the  work  that 
it  must  do  at  each  node  in  the  search  space  of  plans.  The  usefulness  of  lemmas  such  as  LT3  that 
describe  the  effscts  of  actions  on  an  agent’s  belief  state  now  becomes  apparent:  by  summarising  the 
effect  of  actions  on  an  agent’s  beliefs,  they  reduce  the  complexity  of  the  deductions  that  must  be 
performed  at  each  step  in  the  plan.  Further  savings  can  be  realised  by  using  the  method  of  belief 
attachment  described  in  the  previous  section:  from  at  the  ML,  infer  K[Ao,t,f).  Most  of 

the  work  of  figuring  out  Ao’s  future  states  of  knowledge  can  be  performed  by  reasoning  about  H 
at  the  metalevel,  rather  than  K,  and  this  is  considerably  simpler.  Finally,  it  should  be  not-  that 
the  executability  requirement  acts  as  a  filter  on  plans.  Thus  a  reasonable  search  strategy  would  be 
to  first  find  a  plan  that  works  without  taking  into  account  its  executability  (and  hence  the  future 
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belief  states  of  the  planning  agent),  and  then  teit  it  for  exeeutability. 


SJ  Conelailon 

To  raamariie  the  contribution!  of  thii  paper:  we  hare  defined  a  syntactic  approach  to  the 
repreientation  of  knowledge  and  belief  in  whieh  the  key  element  ii  the  identification  of  belief!  with 
provable  exprenioni  in  a  theory  of  the  object  language.  The  technique  of  semantic  attachment  to  the 
intended  interpretation  of  the  metalanguage  provability  predicate  hat  been  advanced  as  a  method 
of  simplifying  proofs  by  directly  modeling  an  agent’s  inference  procedure,  rather  than  simulating  it. 

To  unify  a  formalisation  of  knowledge  and  action,  we  have  shown  how  to  take  Moore's  account 
of  their  interaction  and  formalise  it  within  the  syntactic  framework.  The  benchmark  example  was 
a  presentation  of  a  test  in  which  an  agent  uses  his  knowledge  of  observable  properties  of  the  world 
and  tne  way  actions  affect  the  world  to  discover  the  state  of  an  unobservable  property.  Finally,  we 
pointed  out  how  the  formalisation  could  be  used  in  a  planning  system. 

While  this  paper  is  a  step  towards  showing  that  the  syntactic  approach  can  be  extended  to 
an  adequate  formalisation  of  the  interaction  of  knowledge  and  action,  there  is  still  much  work  to 
be  done  in  constructing  a  practical  planner  for  a  multiagent  environment  that  uses  this  formalism. 
Two  areas  in  particular  are  critical.  First,  a  suitable  system  for  doing  automatic  deduction  in  the 
framework  has  to  be  worked  out.  Although  we  have  advocated  semantic  attachment  as  a  means  of 
simplifying  proofs,  we  have  not  yet  explored  the  problem  of  controlling  a  deduction  mechanism  that 
uses  this  technique.  The  second  area  also  involves  control  issues:  how  can  a  planner  be  designed  to 
search  the  space  of  multiagent  possible  plans  efficiently?  One  of  the  ideas  suggested  by  this  paper 
is  to  derive  lemmas  of  the  form  of  LT 3  that  show  the  effect  of  actions  on  an  agent’s  beliefs.  With 
such  lemmas,  a  planning  system  would  have  already  compiled  the  necessary  results  for  constructing 
new  belief  states  from  previous  ones. 


Appendix  A:  The  Wise  Mu  Pus  tie 


Thii  is  s  solution  to  s  simple  Tsrsion  of  the  wise  mu  puisle,  for  whose  statement  we  quote 
from  [McCarthy  *t.  el.  1980]:  A  king  witting  to  know  which  of  hit  three  wise  men  it  the  wisest, 
paints  white  doti  on  each  of  their  foreheads,  teiis  them  that  at  least  one  spot  is  white,  and  asks 
each  to  determine  the  color  of  hit  spot.  After  a  while  the  smartest  announces  that  hit  spot  it  white 
reasoning  as  follows;  ‘Suppose  mj  spot  were  hlack.  The  second  wisest  of  us  would  then  see  a  black 
and  a  white  ud  would  reason  that  if  his  spot  were  black,  the  dumbest  would  see  two  black  spots  and 
would  conclude  that  his  spot  is  white  on  the  basis  of  the  king’s  assurance.  He  would  hare  announced 
it  by  now,  so  my  spot  must  be  white." 

We  will  simplify  this  puisle  by  having  the  king  ask  each  wise  man  in  turn  what  color  his  spot 
is,  starting  with  the  dumbest.  The  first  two  say  "no,”  ud  the  last  says  that  his  spot  is  white. 

In  formalising  the  puisle,  we  will  take  the  three  wise  mu  to  be  Ao,  A\,  ud  Aa,  in  order  of 
increasing  stupidity.  We  will  reason  about  the  puisle  from  Ao’s  point  of  view,  ud  show  that  Ao 
knows  that  his  spot  is  white  after  hearing  the  replies  of  the  other  two.  We  will  not  be  concerned 
with  the  axiomatisation  of  the  speech  act  performed  by  the  agents;  it  will  be  assumed  that  Ao’s 
model  of  the  world  changes  appropriately  to  reflect  this  new  information. 

There  are  three  situations  in  the  puisle:  the  initial  situation  So,  the  situation  Si  just  after  Aa 
speaks,  ud  the  situation  Sa  just  after  Ai  speaks.  The  frame  axioms  for  these  situation  are  simply 
that  every  agent  knows  what  he  knew  in  the  previous  situation;  these  frame  axioms  are  common 
knowledge. 

We  will  identify  Ao  with  the  ML,  so  the  goal  is  to  show: 


H[St,W[Ao)) 

in  the  ML.  W[d)  is  the  predicate  whose  meaning  is  *a’s  spot  is  white.”  The  initial  conditions  of  the 
problem  are: 
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(1)  W(At)AW(Aa) 

(2)  CFACT(‘W(Ao)VW(Ai)VW(Aa)’) 

(3)  CFACT{‘K[Aa,  So,  ’  W(Ao)’)Vtf  (Aa,  So,  ,~W(Ao)’)’) 

(4)  CFACT('K{A2,  So,  ,W(Al)’)\/fC(A2,  S0,  *~W’(A1)T) 

(5)  K(Ai,  So, 'W{Ao)’)sjK{Au  So,  ’-WfAo)’) 

(8)  CFACT(‘~K(A2,  S0i  ‘W(Aa)’)’) 

(7)  CFACTC~K(A1,Sl;W(Ai)')') 


Line  1  says  that  Ao  observes  white  spots  on  Ai  and  Aa;  line  2  asserts  that  it  is  common  knowledge 
that  at  least  one  spot  is  white.  The  next  two  lines  state  that  it  is  common  knowledge  that  Aa  can 
observe  whether  the  other  two  agent’s  spots  are  white  or  not.  Line  5  says  that  At  knows  the  color 
of  Ao's  spot.  And  the  last  two  lines  express  the  effect  of  the  first  two  agent’s  answers  to  the  king 
on  everyone’s  knowledge.  This  axiomatisation  will  be  sufficient  to  prove  that  Ao  knows  his  spot  is 
white  in  S3. 

The  first  step  in  the  proof  is  to  show  that  A\  knows,  in  situation  Si,  that  either  his  own  or 
Ao’s  spot  is  white;  this  by  reasoning  about  Aa's  answer  to  the  king.  We  will  attach  to  Ai’s  theory 
in  situation  Si  (that  is,  th*(Ai,Si)),  and  do  our  reasoning  there: 


Ai’s  Theory  in  Situation  St 


(8)  ~K-(Aa,Sh,’W(Aa)’) 

(9)  K{A3,  Sb,  *W(Ao)V WUOV W(Aa)’) 

(10)  K(A2,  So,  ‘(~W(Ao)A~W{Ai)) D W(Aa)’) 

(11)  JC(Aa,  So,  ’~W(Ao)A~W(Ai)’)D JC(Aa,  S0,  ’WW) 

(12)  ~tf(Aa,Sb,’~W(Ao)A~W{Ai)’) 


6, semantic  attachment 
3, semantic  attachment 
9 

10,  MP 

8,11  contrapositive 


In  these  lines,  we  have  used  the  fact  that  everyone  knows  that  everyone  knows  common  knowledge 
assertions.  At  line  12,  At  realises  that  Aa  doesn’t  know  that  both  Ao  and  At  lack  white  dots;  if  he 
did,  ha  would  have  announced  the  fact. 

Now  At  uses  the  common  knowledge  that  Aa  can  observe  the  color  of  Ao’s  and  Ai’s  dots  to 
reason  that  one  of  the  latter  has  a  white  dot: 
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Ai’s  Theory  is  Situation  Si 


(13)  K Vir^)*) a* (it,,  So,  ‘-V^Ai)*) 

(14)  tf(Aa,  ^  ,~W(A0)A~W[Ai)') 

(15)  ~tf(Aa,  So,  •-W’fAo)’)  V  ~K(Ao,  S0,  *~W(A1)') 

(16)  ~tf(Aa,$»,‘~;V(Ao)*) 

(IT)  if(A3,SbI*W(Ao)*)VJf(Aa,So,‘~V^Ao)‘) 

(18)  K{Ao,So,'W{AcY) 

(19)  ~IT(Aa,Sb,’~W(A,)’) 

(20)  K(Ao,  So,  'W{A\Y)yK{M,  So,  ^WfA,)’) 

(21)  K{Aa,So,'W(Ai)’) 

(22)  K{A a,  Sb,  W(Ao)*)Vlf(Aa,  S0,  ’H^Ai)*) 

(23)  H{So,‘W{Ao)VWAx)'} 

(24)  W’(Ao)VW(A1) 


[13]:assumption 
[131:13,  PR 

13;  12,14  contradiction 
[16|:auumption 
3, semantic  attachment 
[161:18, IT 
[19]:auumption 
4,iemantic  attachment 
[19):19,20 
15,16,18,19,21 

22, B2 

23,  frame  azioma,Rl 


We  flnt  thow  here  that  As  doeent  know  Ao’>  ipot  ii  black,  or  he  doesn't  know  that  Ai't  ipot  ii 
black  (Une  15).  Auertiona  that  follow  from  auumptioni  are  indicated  by  a  square  bracketing  of  the 
assumption  line  number  in  their  justification.  Next  we  do  an  analysis  by  cases  of  line  15;  in  either 
ease,  line  22  holds:  A«  either  knows  Ao’s  spot  is  white,  or  he  knows  Ai’s  spot  is  white.  From  this  At 
concludes  that  either  ha  or  A«  has  a  white  spot  (line  24).  Note  that  the  frame  axioms  were  needed 
to  show  that  the  W  predicate  doesn’t  change  from  situation  So  to  situation  Si. 

At  this  point  we  are  through  analysing  Ai’s  theory  of  situation  Si,  and  go  back  to  the  ML  to 
reason  about  situation  Sj.  By  lino  5,  At  knows  the  color  of  Ao’s  dot,  so  we  assume  that  he  knows 
it  is  black: 


At  the  Metalerel 


(25)  K(Ai,So,  ‘~W(Ao)’)  [25]:a**umption 

(26)  KiAi.Si.'^WiAo)*)  (25):25, frame  axiom* 

(27)  K(Al,Su^^W{Ae)DW^Al)•)  24, frame  exiomi 

(28)  K(Al,Si,,W(Al)’)  [251:26, 27.A/P 

(20)  ~/T(Ai,  Si,  'W(Ai)')  7,  common  knowledge 

(30)  'wlf(Al,So,‘~W’(i4o),)  25,  28, 20, contradiction 

(31)  K(Ai,Sq,  ‘W^(Ao)’)  5,30 

(32)  H(So,‘W{AoY)  31,  B2 

Under  the  aeiomption  that  At  knowi  Ao’i  «pot  it  black,  we  derive  the  contradiction  of  line*  28  and 
20.  Therefore,  by  line  5,  it  mtut  be  the  cate  that  At  knowi  Ao’i  tpot  to  be  white.  Thit  it  the 
conclution  of  line  32;  tince  thit  it  one  of  Ao’t  belief*,  we  are  done. 
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